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CHAPTER  I 


INTRODUCTION  AND  SUMMARY 


The  U.S.  military  services  have  been  users  of  flight  training 
devices  and  simulators  for  over  half  a  century.  These  training  media, 
known  collectively  as  aircrew  training  devices  (ATDs),  include  cockpit 
famil iarization  and  procedures  trainers,  part-task  trainers,  mission 
trainers,  operational  flight  trainers,  and  weapons  systems  trainers.  In 
recent  years,  use  of  ATDs  has  increased  to  the  point  that  the  devices 
represent  major  aircrew  training  resources,  and  their  effective  and 
efficient  design  and  use  are  matters  of  continuing  concern. 

In  response  to  this  concern,  the  U.S.  Air  Force  has  undertaken  a 
programmatic  study  of  factors  involved  in  their  design  and  use.  This 
project  is  titled  Simulator  Training  Requirements  and  Effectiveness 
Study  (STRES).  The  general  objectives  of  STRES  are  to  define,  describe, 
collect,  analyze,  and  document  information  bearing  on  the  cost  and 
training  effectiveness  of  flight  simulators.  Topic  areas  covered  in  the 
project  are:  fidelity;  instructional  support  features;  simulator 

utilization;  life  cycle  costs;  and  worth  of  ownership.  Products  of  the 
project  are  intended  for  those  who  manage  and  use  simulators  for 
training,  evaluate  simulator  requirements,  and  design,  procure,  and 
maintain  these  devices.  Chapter  II  describes  the  STRES  project  in  more 
detail,  as  well  as  the  approach  followed  in  its  execution. 

This  volume  is  one  of  seven  prepared  during  STRES.  It  addresses 
concerns  related  to  the  utilization  of  ATDs.  Other  volumes  prepared 
during  the  project  are  identified  in  the  description  contained  in 
Chapter  II. 


PURPOSES  OF  THIS  REPORT 

The  purposes  of  this  report  are  to  identify  factors  which  influence 
the  utilization  of  ATDs  and  to  provide  guidance  concerning  ways  of 
enhancing  the  effectiveness  and  efficiency  of  the  use  of  these  devices. 
The  contents  of  this  report  are  based  upon  a  review  of  relevant 
scientific  and  technical  literature,  observation  of  current  ATD 
utilization  practices,  and  interviews  with  personnel  engaged  in  the 
conduct  and  management  of  ATD  training  activities.  The  scope  of  these 
efforts  was  determined  by  the  Statement  of  Work  under  which  the  project 
was  conducted. 

This  report  was  prepared  for  personnel  who  are  responsible  for  the 
use  of  ATDs.  These  personnel  Include  training  program  developers  who 
may  or  may  not  be  trained  in  Instructional  System  Development  (ISO) 
methods,  rated  and  nonrated  ATD  instructors,  flight  and  ATD  training 
supervisors,  unit  management  personnel,  and  personnel  responsible  for 
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assessing  and  controlling  the  quality  of  ATD  training.  Others  for  whom 
this  report  was  prepared  include  personnel  in  higher  management 
positions  who  are  concerned  with  the  effectiveness  of  ATD  training  and 
the  manner  in  which  the  resources  they  manage  are  utilized. 

Based  upon  the  requirements  of  the  Statement  of  Work  and 
information  collected  during  this  project,  perceptions  were  developed  of 
the  informational  needs  of  the  intended  audience  of  this  report.  These 
needs  are  for  a  working  knowledge  of  principles  of  instructional 
technology  relevant  to  effective  and  efficient  ATD  utilization,  and 
guidance  on  when  and  how  to  use  ATDs  of  various  configurations  and 
complexities.  In  addition,  information  is  needed  by  these  personnel 
regarding  the  management  and  training  of  ATD  instructors  and  the 
influencing  of  their  attitudes  toward  training  and  simulation.  A  need 
also  was  found  for  guidance  concerning  the  assessment  of  the 
effectiveness  of  ATD  training  and  the  maintenance  of  ATD  training 
effectiveness  once  it  has  been  established. 


REPORT  OVERVIEW 

Strengths  and  weaknesses  of  ATD  training  programs  observed  during 
this  project  can  be  related  to  how  six  questions  were  being  approached 
and  related  problems  resolved.  First,  was  the  complexity  of  skill 

learning  recognized,  and  did  the  training  program  adapt  ATD  usage  to  the 
basic  requirements  for  effective  and  efficient  skill  training?  Second, 
was  the  structure  of  ATD  training  appropriate  for  the  skills  be  ng 
trained  and  the  capabilities  of  available  devices?  Third,  were  the 

overall  responsibilities  of  instructors  consonant  with  requisites  for 
effective  Instruction,  and  were  they  adequately  prepared  to  teach  and 
exploit  ATD  capabilities?  Fourth,  had  steps  been  taken  to  ensure  the 

development  and  maintenance  of  attitudes  among  ATD  personnel  and 

students  that  are  necessary  for  a  favorable  training  environment. 
Fifth,  had  effectiveness  and  efficiency  of  ATD  training  been  assessed 
and  had  the  findings  been  disseminated  and  used  to  improve  ATD  training? 
And  sixth,  had  steps  been  taken  to  ensure  maintenance  of  training 
quality  after  an  ATD  program  became  operational? 

Complexity  of  Skill  Learning 


By  and  large,  inflight  training  models  governed  ATD  training. 
Students  simply  practiced  skills  in  devices  with  feedback  and  guidance 
provided  according  to  instructors'  intuitive  analyses  of  their  needs. 
Some  of  the  advanced  ATD  capabilities  and  instructional  support  features 
were  used;  but  just  as  commonly,  others  were  not  used  at  all,  either 
because  they  were  perceived  to  reduce  realism  or  because  the  instructor 
had  not  learned  how  to  use  them. 

Within  this  context,  little  attention  was  given  by  ATD  instructors 
to  particular  cue  and  response  discriminations  that  underlie  skill 
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performance,  although  the  status  of  these  discriminations  at  a  given 
point  in  training  should  be  the  basis  for  deciding  whether  to  provide 
feedback  and  guidance,  and  in  what  form.  Many  instructors  could  adapt 
use  of  these  factors  to  students'  needs,  but  more  often  feedback  and 
guidance  were  used  according  to  a  routine  plan  that  did  not  consider 
students'  actual  needs  on  a  moment-to-moment  basis.  Many  ATDs  are 

specifically  designed  to  optimize  use  of  these  factors;  if  the 
contingencies  for  using  feedback  and  guidance  are  not  recognized, 
however,  it  is  not  possible  to  exploit  them. 

The  most  serious  problem  found  in  ATD  training  was  the  failure  of 
most  instructors  to  recognize  the  fundamental  role  of  mediational 
processes  in  all  skill  learning.  Mediation  refers  to  everything  a 

person  does  between  receiving  a  stimulus  and  responding  to  it--deriving 
cue  Information  from  the  stimulus,  interpreting  and  processing  the 
information,  selecting  a  response,  monitoring  the  response,  etc.  Both 
cognitive  and  motor  mediations  are  involved. 

It  is  because  of  mediation  that  complex  learning  is  possible. 

Transfer  of  ATD  training  to  aircraft  performance  depends  entirely  on 
appropriate  forms  of  this  process.  Rather  than  deliberately  emphasizing 
mediational  development  and  use,  however,  ATD  training  was  found  to 

focus  almost  entirely  on  the  overt  actions  of  students.  The  crucial 
mediational  skills  were  left  to  develop  largely  on  their  own. 

Chapters  III  and  IV  are  devoted  to  examining  the  complexities  of 
learning  and  explaining  their  implications  for  ATD  training. 

Structure  of  ATD  Training 

There  are  many  ways  to  structure  specific  aspects  of  ATD  training 
to  provide  effective  and  efficient  instruction.  Nevertheless,  certain 
general  requirements  must  be  met  regardless  of  the  structure  used. 
First,  training  allocated  to  ATDs  must  be  consistent  with  their 
capabilities.  However,  depending  on  the  students'  capacities  to 
substitute  mediation  for  physical  realism,  and  the  extent  to  which 
Instruction  exploits  these  abilities,  a  variety  of  tasks  can  be  trained 
in  ATDs  with  ostensibly  limited  features. 

Second,  tasks  must  be  separated,  and  grouped,  in  ways  that  promote 
effective  and  efficient  learning.  Depending  on  the  task  and 
discriminative  abilities  of  the  student,  particular  training  sessions 
may  need  to  focus  on  part  tasks  or  grouped  tasks.  There  are  ways  to 
decide  which  approach  is  needed,  and  they  are  based  on  the  processes  of 
skill  acquisition.  Similarly,  there  are  ways  to  promote  the  functional 
integration  and  coordination  of  skills,  whether  learned  separately  or 
together. 

Third,  the  duration  and  frequency  of  practice  should  be  such  that 
progress  is  made  during  skill  acquisition  and  the  skills  learned  are 


11 


retained.  Factors  to  be  considered  include  the  need  to  reduce 

interference  on  some  occasions,  and  to  promote  the  accommodation  of 
interference  on  others;  the  amount  of  forgetting  that  occurs  between 

practice  sessions  and  over  time  after  skills  are  learned;  and  the  level 
of  mastery  previously  achieved  by  students.  The  nature  of  individual 
tasks  practiced  in  sequence  is  also  an  important  factor  in  determining 
duration  of  single  ATD  sessions,  especially  for  less  experienced 
students. 

Fourth,  the  effectiveness  of  ATD  training  in  some  skills  can  be 
affected  by  when  it  occurs  relative  to  academic  training  and  aircraft 
experience.  For  some  tasks,  later  ATD  practice  can  provide  concrete 

meanings  for  cues  and  actions  learned  abstractly  in  academic  training; 

in  other  cases,  academic  training  may  be  almost  meaningless  unless 
students  have  already  had  certain  ATD  (or  aircraft)  experiences. 
Similarly,  experiences  In  aircraft  may  need  to  precede  ATD  training  for 
some  tasks  and  follow  it  for  others. 

Chapter  V  examines  bases  for  structuring  ATD  training  programs  with 
respect  to  these  four  considerations. 

ATD  Instructors 


As  a  rule,  the  more  effective  ATD  training  programs  observed  during 
site  visits  were  those  in  which  instructors  had  been  trained  to  teach 
and  were  taught  specifically  how  to  use  the  ATD  and  its  instructional 
support  features.  These  programs  also  minimized  non-instructional 
duties  that  would  interfere  with  teaching  performance.  In  most 
effective  programs,  ATD  instructors  were  considered  an  elite  group  by 
their  peers  and  students.  To  the  extent  the  reverse  of  these  conditions 
was  true,  training  programs  appeared  to  have  been  negatively  affected. 

Chapter  VI  discusses  the  effects  of  the  selection,  training,  and 
management  of  instructional  personnel  on  the  quality  of  ATD  training. 

Attitudes  Toward  ATD  Training 

Attitudes  of  instructors  toward  ATD  training  can  be  of  critical 
importance  for  the  success  of  an  ATD  program.  Not  only  do  instructor 
beliefs  regarding  device  training  determine  how  devices  are  employed, 
but  they  affect  the  attitudes  of  students  toward  the  value  of  ATD 
practice.  The  problem  of  attitudes  goes  beyond  the  instructors  and 
students,  however.  Just  as  students  "model"  instructors'  opinions,  so 
instructors  reflect  the  beliefs  of  their  peers  and  supervisors.  The 
problem  of  fostering  favorable  attitudes  toward  ATD  instruction  must  be 
faced  at  the  organizational  level. 

Chapter  VII  discusses  causes  for  the  conditions  leading  to 
favorable  and  unfavorable  attitudes  toward  ATD  training.  These  causes 
and  conditions  relate  to  how  design  requirements  for  ATDs  are 
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determined,  the  manner  of  Introduction  of  an  ATD  to  a  training  unit,  and 
how  ATD  training  is  conducted  and  managed.  The  need  for  an  atmosphere 
of  professionalism  of  instructors  qua  instructors  is  also  explained. 

Assessment  of  Training  Effectiveness 

Formal  assessments  of  the  training  effectiveness  of  operational  ATD 
training  programs  are  practically  non-existent.  Therefore,  there  are  no 
hard  data  to  identify  shortcomings  of  programs,  or  to  convince  doubting 
Instructors  of  the  value  of  ATD  training.  In  fact,  if  one  had  to  rely 
on  existing  documentation  of  training  effectiveness,  he  could  not  even 
establish  that  ATD  programs  are  worth  the  effort. 

Formal  assessment  of  these  programs  requires  sophistication  in 
evaluation  methodology.  No  attempt  is  made  to  teach  this  knowledge. 
However,  Chapter  VIII  is  devoted  to  explaining  what  Is  Involved  to 
provide  guidance  to  personnel  who  must  judge  the  adequacy  of  evaluation 
studies. 

Maintaining  ATD  Training  Effectiveness 

Given  an  effective  ATD  program,  its  quality  can  be  maintained  only 
if  it  adjusted  quickly  to  modifications  in  training  requirements  and  in 
design  of  target  aircraft.  Good  training  practices  must  be  maintained 
and  not  allowed  to  evolve  in  undesirable  directions  as  training 
personnel  are  replaced  or  student  quality  changes.  Attitudes  toward  ATD 
training  may  also  change  for  the  worse  over  time,  and  management  may 
Increasingly  neglect  the  program  as  its  novelty  wears  off. 

Problems  such  as  modifications  in  training  requirements  and 
aircraft  should  be  resolved  posthaste.  Others,  especially,  those  leading 
to  a  gradual  deterioration,  may  not  be  noticeable  until  serious  damage 
has  occurred  unless  quality  is  monitored  and  corrective  action  taken  on 
a  regular  basis.  Chapter  IX  discusses  these  problems. 
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CHAPTER  II 


THE  STRES  PROGRAM 


Aircrew  training  Is  an  expensive  and  time  consuming  endeavor.  At 
one  time  or  another,  virtually  every  known  training  method  and  medium  has 
been  used  to  develop  operationally  ready  aircrews  and  to  maintain  their 
skill  levels.  To  meet  these  training  needs  in  a  cost  effective  manner, 

the  U.S.  military  has  shown  increased  interest  in  the  use  of  simulators 

and  other  ATDs.  Recent  requirements  to  economize  on  aircraft  fuel  used 
for  training  have  provided  strong  impetus  for  this  interest,  but  other 
factors  have  contributed  as  well.  These  other  factors  include 

increasingly  congested  airspace,  safety  during  training,  cost  of 
operational  equipment  used  for  training,  and  a  desire  to  capitalize  on 
training  opportunities  that  simulators  provide  for  skills  that  cannot  be 
trained  effectively,  safely,  or  economically  in  the  air. 

Because  of  the  advantages  simulation  can  offer  over  other  aircrew 
training  media,  it  is  current  Air  Force  policy  that  ATDs  will  be  used  to 
the  fullest  extent  to  improve  readiness,  operational  capability,  and 
training  efficiency.  Implementation  of  this  policy  requires  specific 

technical  guidance.  Information  upon  which  to  base  that  guidance  is 
sparse,  however,  and  the  Information  that  does  exist  is  not  always 
available  to  those  who  need  it.  The  Simulator  Training  Requirements  and 
Effectiveness  Study  (STRES)  was  conceived  as  a  means  of  identifying  and 
making  available  the  existing  information  "elated  to  simulator  training 
In  furtherance  of  relevant  Air  Force  policies.  The  base  of  information 
thus  assembled  would  provide  guidance  for  the  enhancement  of  present 
training,  as  well  as  for  the  focus  of  research  and  development  needed  to 
enhance  future  simulation-based  training. 


STRUCTURE  OF  THE  STRES  PROGRAM 

The  primary  objectives  of  STRES,  as  described  in  the  contract 
Statement  of  Work  for  the  present  efforts,  are  to  define,  describe, 
collect,  analyze  and  document  information  bearing  on  four  key  areas. 
These  areas  are: 

•  Criteria  for  matching  training  requirements  with  simulator 

fidelity  features; 

•  Principles  of  effective  and  efficient  utilization  of  simulators 
to  accomplish  specific  training  requirements; 

•  Criteria  for  matching  simulator  instructional  features  with 

specific  training  requirements;  and 

•  Models  of  factors  influencing  the  cost  and  the  worth  of  ownership 
of  simulation  devices. 
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The  Air  Force  plan  for  accomplishing  these  objectives  involves  a 
four-phase  effort-  Phase  I,  which  was  concluded  prior  to  the  initiation 
of  the  present  study,  was  an  Air  Force  planning  activity  that  structured 
the  total  effort  so  that  operationally  meaningful  simulator  training 
issues  would  be  addressed  on  a  priority  basis.  Phase  II,  the  effort 
described  in  the  series  of  reports  Identified  below  of  which  the  present 
report  is  a  part,  was  a  29  month  study  that  involved  collecting, 
integrating,  and  presenting  currently  available  scientific,  technical, 
and  operational  information  applicable  to  specific  aircrew  training 
issues.  Phase  II  also  involved  the  identification  of  research  and 
development  efforts  needed  to  enhance  future  simulator  training.  Phase 
II  was  conducted  by  a  team  composed  of  Canyon  Research  Group,  Inc., 
Seville  Research  Corporation,  and  United  Airlines  Flight  Training  Center. 
Phase  III  is  planned  to  be  a  research  activity  that  will  provide 
additional  information  on  important  simulation  and  simulator  training 
related  questions  that  cannot  be  answered  with  assurance  with  the 
currently  available  data.  Finally,  building  on  Phases  II  and  III,  Phase 
IV  will  be  an  Air  Force  effort  to  integrate  findings,  publish  relevant 
information,  and  provide  for  updating  of  the  knowledge  base  as  new 
information  becomes  available. 

A  tri -service  STRES  Advisory  Team  was  formed  to  help  guide  STRES. 
The  team  has  participated  in  two  ways.  One  was  to  assist  in  the  Phase  I 
program  planning.  The  second  has  been  to  provide  guidance  and  evaluative 
feedback  during  Phase  II  to  ensure  that  products  of  the  phase  would  be 
operationally  relevant  and  useful.  Both  operational  users  of  ATDs  and 
the  research  community  were  represented  on  the  Advisory  Team. 

A  principal  task  of  the  Advisory  Team  was  to  participate  in  the 
development  of  objectives  and  guidelines  for  the  conduct  of  the  Phase  II 
technical  effort.  As  a  focus  for  those  efforts,  a  set  of  “high  value" 
operational  tasks  was  identified.  The  tasks  selected  were  those  for 
which  potential  ATD  training  benefits  were  judged  to  be  greatest,  and  for 
which  Information  on  ATD  desicn,  retrofit,  use,  and  worth  was  believed  to 
be  incomplete  or  lacking.  These  tasks  also  provided  a  focus  for 
identification  of  questions  and  issues  reflecting  the  informational  needs 
of  operational  personnel  that  were  to  be  addressed  during  Phase  II 
efforts.  The  high  value  tasks  identified  by  the  Advisory  Team  are: 

•  Individual  and  formation  takeoff  and  landing; 

•  Close  formation  flight  and  trail  formation, 
both  close  and  extended; 

•  Aerobatics; 

•  Spin,  stall,  and  unusual  attitude  recognition, 
prevention,  and  recovery; 

•  Low  level  terrain  following; 

•  Air  refueling; 
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•  Air-to-air  combat,  both  guns  and  missiles; 

•  Ai r- to-ground  weapons  delivery. 


SOURCES  OF  INFORMATION 

Information  from  two  kinds  of  sources  was  collected  during  Phase  II 
to  address  the  objectives  of  STRES.  One  kind  of  source  was  the 
professional  and  techical  literature.  This  literature  included  books, 
conference  proceedings,  professional  journals,  research  reports,  military 
manuals  and  regulations,  and  policy  statements.  The  second  kind  of 
source  was  the  military  and  civilian  personnel  whose  experiences  related 
to  the  objectives  of  the  study.  Information  was  obtained  from  these 
personnel  during  visits  to  the  organizations  to  which  they  were  assigned. 

Literature  Review 


Computer  searches  were  made  at  the  outset  to  identify  literature 
relevant  to  all  facets  of  the  Phase  II  effort.  In  addition,  each  STRES 
team  member  was  responsible  for  identifying  documents  pertinent  to  his 
responsibilities  that  may  have  been  missed  in  the  computer  searches.  In 
these  individual  efforts,  articles  pertinent  to  the  various  activities  of 
colleagues  were  regularly  encountered.  Each  investigator  was  aware  of 
the  Informational  needs  of  his  colleagues,  and  frequent  communication 
among  team  members  assured  that  colleagues  would  be  apprised  of  articles 
of  potential  value  to  their  tasks.  Hence,  the  search  for  literature  of 
concern  to  the  preparation  of  a  given  volume  of  the  STRES  report  series, 
while  systematically  compiled  by  those  specifically  responsible  for  that 
volume,  was  expanded  through  the  efforts  of  the  entire  team. 

To  provide  integration  and  focus  to  these  literature  search  efforts, 
one  group  of  the  STRES  team  was  specifically  responsible  for  identifying 
articles  of  potential  interest  to  all  team  efforts,  as  well  as  for 
preparing  comprehensive  abstracts  of  selected  documents  that  appeared 
particularly  valuable.  These  abstracts  appear  in  a  separate  volume  of 
the  STRES  report  series. 

More  than  1100  documents  were  identified  during  these  efforts  as 
potentially  relevant.  These  were  further  screened  according  to  the 
currency  and  completeness  of  information  provided  and  the  integrity  of 
the  experimental  and  analytic  methods  used.  As  a  result  of  this 
screening,  approximately  400  documents  were  found  to  be  useful  for  STRES 
purposes. 

Site  Visits 

A  considerable  body  of  information  was  also  obtained  from 
organizations,  both  government  and  commercial,  whose  personnel  are 
involved  in  the  design,  procurement,  evaluation,  management,  and  use  of 
ATDs.  ATD  manufacturers,  research  and  development  agencies,  and  a 
commercial  airline  were  visited  in  addition  to  Air  Force,  Army,  Navy,  and 
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Coast  Guard  military  training  sites.  At  each  organization,  extensive 
data  were  obtained  through  observations,  Interviews,  and  document 
reviews.  The  training  organizations  visited  and  the  topics  of  primary 
Interest  at  each  are  listed  In  Table  1.  Table  2  lists  non-training 
organizations  that  were  visited  and  corresponding  interest  topics. 


TABLE  1.  TRAINING  SITES  INCLUDED  IN  PROGRAM  SURVEYS 


Sites  and  Units 

Topics  of  Interest 

Altus  AFB,  OK  (MAC) 

443rd  Military  Airlift  Wing 

C-5  transition  training 

Castle  AFB,  CA  (GAC) 

93rd  Bomb  Wing 

KC-135/B-52  transition 
training 

Denver,  CO 

United  Airlines 

Flight  Training  Center 

DC-10/8-737/B-747  transi¬ 
tion  and  continuation 
training 

Eglln  AFB,  FL 

33rd  Tactical  Fighter  Wing 

F-4  continuation  training 

Fort  Rucker,  AL 

U.S.  Am\y  Aviation  Center 

UH-1  and  CH-47  under¬ 
graduate  and  transition 
training 

Langley  AFB,  VA  (TAC) 

1st  Tactical  Fighter  Wing 

F-15  continuation  training 

Mobile,  AL 

U.S.  Coast  Guard  Aviation 

Training  Center 

HH-3/HH-52  transition  and 
continuation  training 

NAS.  Cecil  Field,  FL 

VP-174  and  Light  Attack 

Air  Wing  One 

A-7E  transition  and 
continuation  training 

NAS,  Jacksonville,  FL 

VP-38  and  Patrol  Wing  Eleven 

P-3C  transition  and 
continuation  training 

Plattsburgh  AFB,  NY  (SAC) 

380th  Bomb  Wing 

FB-111  transition  training 

Reese  AFB,  TX  (ATC) 

64th  Flying  Training  Wing 

T-37/T-38  undergraduate 
pilot  training 

Tinker  AFB,  OK  (TAC) 

552nd  Airborne  Warning 
and  Control  Wing 

E-3A  transition  and 
continuation  training 
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TABLE  2.  SITES  VISITED  FOR  MANAGEMENT,  RESEARCH, 
DEVELOPMENT,  ENGINEERING  AND  COST  SURVEYS 


Sites  and  AgencieT 


Washington,  DC 
Pentagon 

Headquarters,  USAF 

Randolph  AFB,  TX 
Headquarters,  ATC 


Langley  AFB,  VA 
Headquarters,  TAC 


Eglin  AFB,  FL  (TAC) 

Tactical  Air  Warfare  Center 

Luke  AFB,  AZ  (TAC) 

4444th  Operational  Training 
Development  Squadron 

Williams  AFB,  AZ 

Air  Force  Human  Resources 

Laboratory  (AFHRL/OT) 

Wright-Patterson  AFB,  OH 
Air  Force  Human  Resources 
Laboratory  (AFHRL/LR) 

Fort  Rucker,  AL 

US  Army  Research  Institute  for 
the  Behavioral  and  Social  Sciences 

Langley,  VA 

NASA  Langley  Research  Center 

St.  Louis,  MO 
McDonnel  Douglas  Corp. 

Binghamton,  NY 
Singer-Link  Corp. 

Orlando,  FL 

Navy  Training  Analysis  and 
Evaluation  Group 


_ Topics  of  Interest 

Management  of  Air  Force  ATD 
resources,  and  life  cycle  costs 


Management  of  the  use  of  ATDs  in 
undergraduate  pilot  training,  and 
life  cycle  costs 

Management  of  the  use  of  ATDs  in 
fighter  aircrew  training,  devel¬ 
opment  of  ATD  requirements,  and 
life  cycle  costs 

Procurement,  development  and 
evaluation  of  ATDs 

Development  of  training  and  ATD 
requirements 


ATD  research 


ATD  research 


ATD  research 


ATD  research 


ATD  design  and  research 


ATD  design,  procurement  and 
evaluation 

ATD  research  and  life  cycle  costs 
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TABLE  2.  Continued 


Sites  and  Agencies 

Topics  of  Interest 

Orlando,  FL 

Naval  Training  Equipment  Center 

ATD 

research  and  life  cycle  costs 

San  Diego,  CA 

Navy  Personnel  Research 

ATD 

research  and  life  cycle  costs 

and  Development  Center 

Orlando,  FL 

US  Army  Project  Manager  for 
Training  Devices  (PM  TRADE) 

ATD 

research  and  life  cycle  costs 

Hill  AFB,  UT  (AFLC) 

ATD 

life  cycle  costs 

Hollomon  AFB,  NM  (AFTEC) 

ATD 

life  cycle  costs 

Luke  AFB,  AZ  (TAC) 

ATD 

life  cycle  costs 

Offutt  AFB,  NE  (SAC) 

ATD 

life  cycle  costs 

Scott  AFB,  IL  (MAC) 

ATD 

life  cycle  costs 

Travis  AFB,  CA  (MAC) 

ATD 

life  cycle  costs 

Williams  AFB,  AZ  (ATD) 

ATD 

life  cycle  costs 

Wright-Patterson  AFB,  OH  (ASD) 

ATD  engineering  and  life  cycle 
costs 

Specific  objectives  of  the  interviews  and  other  data  collection 
efforts  varied,  depending  on  the  type  of  organization  visited  and  the 
purpose  of  the  visit.  Manufacturers  and  research  and  development 
agencies  were  visited  to  assess  current  and  projected  technology  and  to 
review  ongoing  and  planned  efforts  bearing  on  STRES  program  objectives. 
ATD  using  organizations  were  visited  to  obtain  a  variety  of  information 
related  to  types  and  effectiveness  of  training  accomplished,  uses  of 
various  types  of  devices  in  accomplishing  the  training,  ATD  design 
characteristics,  worth  of  ATD  ownership,  and  ATD  life  cycle  costs. 


STRES  PHASE  II  REPORTS 

Seven  reports  were  prepared  to  document  Phase  II  efforts  and 
findings: 


Caro,  P.w.,  Sheinutt,  J.d.,  4  Spears,  w.D.  Aircrew  Training 

Jevices:  Utilization.  AFhRl-TR-80-35.  wright-Patterson 

AFd,  OH:  Logistics  anu  Tecnnical  Training  oivision,  Air 
Force  Human  Kesources  Laooratory,  January  1981. 

Semple,  C.A.,  Hennessy,  K.T.,  dangers,  M.s.,  Cross,  d.K.,  deitn, 
H.H.,  &  McCauley,  M.E.  Aircrew  Training  uevices:  Fiaelity 
Features.  AFHRL-TK-au-3t>.  Wrignt-Patterson  A  Fb,  uh: 

Logistics  ana  Tecnnical  Training  uivision,  Air  Force  Human 
Kesources  Laooratory,  January  1981. 

Semple,  C.A.,  Cotton,  J.C.,  4  dullivan,  u.J.  Aircrew  Training 

□evices:  instructional  Support  Features.  AFHRl-Tk-8U-S8. 

Wright-Pat terson  AFa,  UH:  Logistics  ana  Technical  Training 
Oivision,  Air  Force  human  Kesources  Laooratory,  January  1981. 

Alloee,  K.E.,  4  bempie,  C.A.  Aircrew  Training  uevices:  Life 

Cycle  Cost  ang  Worth  of  Ownership.  AFHRl-TR-80-34. 
Wright-Patterson  AF8,  OH;  Logistics  ana  Technical  Training 
Division,  Air  Force  Human  Resources  Laooratory,  January  1981. 

Prophet,  W.W.,  Sheinutt,  J.b.,  4  Spears,  w.D.  Simulator  Training 
Requirements  ang  Effectiveness  Stuay  CSTREST;  Future 
Research  Plans.  AFHRl-TR-80-37.  wright-Patterson  AFd,  UH: 
Logistics  ana  Technical  Training  uivision,  Air  Force  Human 
Resources  Laooratory,  January  1981. 

Spears,  w.D.,  Sheppard,  H.J.,  Roush,  M.D.,  4  Richetti,  C.L. 

Simulator  Training  Requirements  ana  Effectiveness  Study 
(STRES):  Aostract  Biolioqraphy .  AFHKL-TR-80-38. 

Wright-Patterson  AFd,  OH:  Logistics  ana  Technical  Training 
Division,  Air  Force  Human  Resources  Laooratory,  January  1981. 

Semple,  C.A  Simulator  Training  Requirements  and  Effectiveness 

Study  (STKESj:  Executive  SummaryT  AFHRl-TR-80-63. 

Wright-Patterson  AFd,  OH:  Logistics  ana  Technical  Training 
Division,  Air  Force  Human  Resources  Laooratory,  January  j98l. 


The  content  of  the  first  four  of  these  reports,  l.e.,  ATD  fidelity, 
instructional  features,  utilization,  and  cost  and  worth  of  ownership,  is 
interrelated.  As  an  aid  to  the  reader  in  accessing  related  information, 
these  four  reports  are  cross-referenced.  Within  a  single  volume,  other 
chapters  where  related  information  is  discussed  are  referenced.  When 
the  cross-referenced  information  is  in  another  volume,  that  volume  is 
identified  by  abbreviated  title  (Fidelity,  Instructional  Features, 
Utilization,  or  Cost)  as  well  as  by  chapter.  For  example.  Utilization, 
Chapter  IV,  would  indicate  that  related  information  will  be  found  in 
Chapter  IV  of  the  report  titled  Utilization  of  Aircrew  Training  Devices. 
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As  an  additional  aid  to  the  reader,  the  Executive  Summary  volume 
reproduces  the  tables  of  content  of  all  four  volumes  to  provide  a 
i. or, sol i dated  listing  of  topics  addressed  in  each. 


APPROACH  TO  THE  STUDY  OF  ATD  UTILIZATION 

The  present  report  documents  the  findings  of  that  portion  of  the 
STRES  Phase  II  effort  having  to  do  with  utilization  of  aircrew  training 
devices.  The  approach  employed  was  described  above,  i.e.,  to  compile 
information  obtained  through  literature  review  and  during  visits  to 
training,  manufacturing,  and  research  and  development  organization;  to 
analyze  that  information;  and  to  document  it  for  the  use  of  persons 
involved  in  the  evaluation,  management,  and  utilization  of  ATDs. 
Considerations  of  particular  concern  in  the  conduct  of  this  portion  of 
the  STRES  effort  are  treated  in  the  discussion  that  follows. 

Literature  Review 


Articles  identified  during  the  literature  searches  described  above 
were  screened  for  relevance  to  ATD  utilization.  Relevance  was  defined 
very  broadly,  however,  for  the  perspective  needed  to  answer  questions 
related  to  ATD  utilization  frequently  required  knowledge  of  broader 
issues  such  as  the  design  of  ATDs  and  state-of-the-art  simulator 

technology.  Also,  information  concerned  primarily  with  phenomena  of 
learning  and  related  instructional  practices,  both  theoretical  and 
applied,  was  needed  in  the  assessment  of  reported  ATD  utilization  per  se. 
Of  particular  interest  in  the  learning  literature  were  conditions  and 
practices  that  affect  transfer  of  ATD  training  to  aircraft,  and  the 
retention  of  skills  acquired  in  ATD  training  over  time. 

Site  Visit  Activities 

Activities  of  the  utilization  study  team  at  the  training  sites 
Identified  in  Table  1  included  inspections  of  available  simulators  and 
related  training  aids,  and  observation  of  demonstrations  of  pertinent 
aspects  of  their  capabilities  and  uses.  The  major  portion  of  the  time, 
however,  was  spent  in  intensive  interviews  with  representative 
Individuals  from  all  aspects  of  the  local  training  program.  These 

personnel  included  flight  and  simulator  instructors,  pilots  undergoing 
training,  training  program  developers,  and  management  personnel.  A 
uetalled  interview  guide  was  used  (see  Appendix  A),  and  notes  were 
recorded  during  and  following  the  interviews. 

Detailed  Information  relating  specifically  to  ATDs  included:  (1) 
training  objectives;  (2)  training  methods  and  techniques;  (3)  training 
effectiveness;  (4)  ATD  and  flight  training  program  development  and 
Introduction;  and  (5)  nature  and  qualifications  of  training  personnel. 

As  revealed  in  the  interview  guide,  information  pertaining  to  the 
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overall  training  programs  was  also  acquired  so  that  the  utilization 
practices  applying  to  ATOs  per  se  could  be  put  into  perspective. 

Interviews  with  key  personnel  were  also  the  principal  information 
gathering  technique  employed  during  visits  to  the  sites  identified  in 
Table  2.  The  interview  guides  prepared  for  use  at  the  training  sites 
served  as  checklists  for  topics  to  be  addressed  at  each  agency.  The 
thrust  of  these  visits,  however,  was  to  obtain  information  about 
research  in  progress  or  planned,  advanced  simulation  technology,  and 
present  and  anticipated  regulatory  requirements  and  policies  that  could 
contribute  to  improved  future  ATD  training  programs.  Information  gained 
from  these  sources  also  broadened  the  perspective  from  which  practices 
at  training  sites  could  be  viewed. 

The  Problem  of  the  Diversity  of  Information 

A  major  goal  of  the  site  visits  and  associated  observations  and 
Interviews  was  the  identification  of  characteristics  of  effective  and 
efficient  ATD  programs.  These  characteristics  could  include  anything 
that  promotes  quality  training,  i.e.,  what  is  done,  how  It  is  done,  and 
the  conditions  under  which  It  Is  done.  The  scope  of  such  a  task  was 
considerable.  It  ranged  from  determining  how  training  requirements  fo- 
ATDs  are  established  and  the  quality  of  their  fulfillment,  to 
understanding  purely  administrative  problems  such  as  pipeline  flows  of 
students,  instructor  and  student  work  load  assignments,  and  provisions 
for  ATD  maintenance.  In  between  were  issues  concerning  training  program 
design,  preparation  of  instructional  personnel,  and  the  day-to-day 
implementation  of  instruction. 

The  scope  of  the  task  was  further  enlarged  by  the  variety  of  types 
of  ATDs  that  had  to  be  considered.  The  skills  to  be  taught  using  them 
range  from  piloting  a  high  performance  fighter  to  operating  a  boom  for 
air-to-air  refueling.  The  devices  themselves  included  mock-ups, 
sophisticated  high-fidelity  OFTs/WSTs,  and  devices  that  fall  between 
these  extremes. 

Bases  for  Evaluating  Training  Programs  and  Practices 

With  few  exceptions,  the  ATD  training  programs  observed  during  the 
present  project  had  not  been  subjected  to  empirical  evaluation  to 
determine  their  effectiveness.  Studies  to  assess  the  effects  of 
specific  program  features  or  training  practices  In  those  programs  were 
non-existent.  Determining  which  of  these  ATD  programs  and  practices 
were  effective,  and  which  were  not,  presented  a  particularly  difficult 
problem  for  the  study  team. 

Reviews  of  the  literature  did  not  provide  a  sufficient  basis  for 
making  the  required  effectiveness  determinations.  Empirical  studies  of 
the  effectiveness  of  Individual  aircrew  training  programs  or  of  the 
contribution  of  ATDs  used  in  such  programs  are  rare.  Host  of  the 
studies  that  have  been  reported  were  based  upon  rational  criteria  rather 
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than  hard  data,  or  they  involved  comparisons  of  al  ernate  curricula  and 
have  relatively  little  generality  beyond  the  specific  device/program 
combination  studied.  Hence,  the  value  of  the  research  literature  was 
limited  primarily  to  suggesting  factors  that  could  potentially  affect 
training  outcomes. 

Objective  criteria  for  training  effectiveness  can  be  defined  easily- 
-e.g.,  the  extent  to  which  measureable  course  objectives  are  met;  the 
extent  to  which  ATD  training  transfers  to  aircraft  performance  as 
determined  by  initial  skills  in  the  aircraft  or  savings  in  post¬ 
transition  training  efforts;  and  the  extent  to  which  skills  acquired  is 
ATDs  are  retained  or  the  amount  of  refresher  practice  required  to 
maintain  or  reacquire  necessary  skill  levels.  These  and  other  criteria 
can  be  applied,  and  methods  for  their  application  are  well  known. 
However,  objective  data  corresponding  to  such  criteria  were  not  available 
for  most  of  the  ATD  training  programs  surveyed  during  the  present  study. 

In  the  absence  of  adequate  objective  data,  other  bases  had  to  be 
employed  to  assess  the  effectiveness  of  ATD  training  programs  and 
practices.  Because  of  the  limited  opportunities  of  the  study  team  to 
observe  and  analyze  each  program  surveyed,  it  was  necessary  to  rely 
heavily  upon  the  judgments  and  observations  of  personnel  who  were 
Involved  in  those  programs  concerning  the  effectiveness  of  each.  To 
guard  against  possible  positive  or  negative  biases  of  these  personnel, 
however,  guidelines  were  established  that  would  provide  safeguards 
against  uncritical  acceptance  of  the  reports  of  those  personnel. 
Following  these  guidelines,  the  i  ivesti gators  formulated  their  own 
judgments  of  training  effectiveness  and  it  is  these  judgments  that 
provide  a  basis  for  the  evaluations  reported  in  subsequent  chapters  jf 
this  report. 

The  guidelines  were  stated  in  the  form  of  six  questions.  The 
answers  to  these  questions  determined  the  confidence  that  could  be  placed 
in  judgments  and  observations  of  the  persons  interviewed  and  provided  a 
basis  for  the  interviewers'  own  assessments  of  effectiveness.  First,  was 
there  a  consensus  of  beliefs  among  personnel  in  a  given  program  regarding 
the  effectiveness  of  ATD  training  generally  as  well  as  with  respect  to 
particular  aspects  of  their  programs?  Second,  could  individual  persons 
support  their  beliefs  logically,  or  with  formal  or  informal  observations, 
by  relating  their  judgments  to  specific  ATD  capabilities  and  training 
practices?  Third,  did  the  ATD  capabilities  and  training  practices  cited 
In  support  of  interviewees'  judgments  actually  exist  at  the  training 
site?  Fourth,  when  viewed  across  sites,  were  there  consistent  patterns 
of  judgments  of  training  effectiveness  associated  with  types  of  ATD 
capabilities  and  training  practices?  Fifth,  had  research  shown  the  ATD 
capabilities  or  training  practices  at  issue  to  have  the  effects 
attributed  to  them?  And  sixth,  are  the  presumed  results  of  training 
consistent  with  what  could  be  expected  with  the  practices  used? 
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In  addressing  these  questions,  the  investigators  weighted  the  sixth 
question  more  heavily  than  any  other  single  question.  In  doing  so,  they 
drew  on  their  knowledge  of  the  learning  process  and  instructions, 
technology.  Numerous  laboratory  studies  of  learning  have  been  reported 
each  year  for  several  decades.  Various  authors  have  presented  digests  of 
these  findings  adapted  to  ATD  training  (e.g.,  Miller,  1953;  Wheaton, 
Fingerman,  Rose,  and  Leonard,  1976;  Wheaton,  Rose,  Fingerman,  Korotkin, 
and  Holding,  1976).  The  instructional  principles  summarized  by  these 
authors  guided  the  investigators  in  evaluating  the  training  practices 
they  observed,  in  evaluating  the  beliefs  of  the  instructional  personnel 
they  interviewed  regarding  the  training  effectiveness  of  those  practices, 
and  in  reaching  the  conclusions  contained  in  this  report. 

Furthermore,  the  questions  were  useful  primarily  for  identifying 

possibly  nonvalid  judgments  of  the  persons  interviewed,  rather  than  for 
confirming  valid  ones.  For  example,  when  instructors  at  one  training 
organization  blamed  poor  ATD  results  on  particular  device  shortcomings, 
but  at  another  site  good  results  had  been  obtained  with  an  equivalent  or 
inferior  device,  little  stock  was  put  in  arguments  that  rested  on  assumed 
values  of  the  ATD  capabilities  at  issue.  Or,  if  an  ATD  feature,  e.g., 
platform  motion  (or  lack  of  it),  was  cited  as  the 
reason  for  training  success  (or  failure),  the  research  literature  was 
consulted  to  see  if  such  an  inference  was  tenable. 

All  in  all,  opinions  of  those  responsible  for  training  generally 
appeared  well  conceived;  and,  in  the  absence  of  hard  data  substantiating 
the  effectiveness  or  ineffectiveness  of  a  particular  program  or  practice, 
their  opinions,  qualified  as  described,  provided  meaningful  bases  for 
ascertaining  effectiveness. 

ATD  Utilization  Rates 

At  the  beginning  cf  this  study,  it  was  planned  to  collect  data  on 
rates  of  utilization  of  ATDs  sc  that  relationships  between  utilization 
rates  and  factors  such  as  training  effectiveness,  device  fidelity,  and 
instructional  features  could  be  studied.  This  plan  was  dropped,  however, 
because  available  ATD  utilization  rate  data  were  suspect  and  cculd  riot  be 
used  for  formal  analyses.  Specifically,  the  single  most  important  factor 
found  to  influence  rates  of  simulator  utilization  was  direction  from  an 
authority  that  a  device  be  used  at  a  specified  minimum  rate.  Instances 
were  found  in  which  mandated  utilization  rates  were  being  achieved 
through  practices  that  could  net  possibly  yield  effective  training,  e.g., 
logging  device  time  while  engaged  in  briefing/debriefing  activities, 
taking  coffee  breaks,  and  performing  maintenance  functions.  In  one 
instance,  an  individual  participated  in  a  survey  interview  while 
reportedly  logging  simulator  time  in  another  building.  The  unit  cc  which 
he  was  assigned,  as  well  as  some  other  units  surveyed  during  the  present 
study,  apparently  viewed  use  of  a  simulator  as  a  "square  to  be  filled"  in 
their  records,  and  utilization  rates  at  mandated  levels  were  being 
recorded  through  questionable  practices  when  necessary. 


The  result  was  that  official  utilization  rates  approached  100%  at 
all  training  sites,  and  it  was  not  possible  to  estimate  the  proportion 
of  time  ATDs  were  used  productively.  There  was  ample  evidence  of 
beneficial  use,  however.  Many  instances  were  found,  even  at  sites  where 
abuses  occurred,  of  utilization  practices  that  were  clearly  productive 
In  terms  of  judged  training  effectiveness  and  efficiency.  Thus, 
observations  of  ATD  utilization  focused  on  quality  rather  than  quantity 
of  usage,  and  evaluations  were  made  according  to  the  criteria  discussed 
in  the  preceding  section. 
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CHAPTER  III 


THE  LEARNING  PROCESS  AND  ATD  UTILIZATION 


A  well  designed  training  program  can  be  the  difference  between  a 
marginal  aircrewman  and  an  excellent  one,  or  between  an  unnecessarily 
expensive  training  program  and  one  that  is  economical  and  efficient.  To 
achieve  these  advantages,  a  training  program  must  include  a  large  nunfcer 
of  features  that  foster  effective  and  efficient  instruction.  Because  a 
major  factor  in  the  effectiveness  of  training  is  the  learner  himself,  no 
single  program  feature,  and  probably  few  combinations  of  several,  are 
likely  to  be  crucial  to  a  program's  success  or  failure.  However,  taken 
collectively,  they  will  have  a  significant  impact  on  the  program's 
quality  and  efficiency. 

ATDs  can  fill  a  major  role  both  in  achieving  excellence  In  aircrew 
training  and  in  reducing  cost.  In  most  training  programs  surveyed  during 
this  study,  ATDs  were  valued  primarily  for  the  latter  reason.  They  are 
generally  cheaper  to  use  than  aircraft,  so  when  they  could  be  substituted 
for  aircraft  in  training,  economy  resulted.  However,  properly  used,  ATDs 
can  also  be  more  effective  than  aircraft  for  many  types  of  training. 
Furthermore,  even  when  their  value  depends  only  on  reduced  cost, 
consideration  should  still  be  given  to  practices  that  will  maximize 
training  effectiveness  and  efficiency  and  thereby  reduce  costs  even 
further. 

The  intent  of  this  chapter,  as  well  as  Chapter  IV,  is  to  explore 
learning  principles  that  can  enable  training  program  developers  to  make 
proper  instructional  use  of  ATDs.  These  principles  are  conceptual  in 
nature  and  cannot  be  given  simply  as  categorical  rules  that  state  what  an 
Instructor  should  do  or  when.  Rules  stating  what  to  do  in  a  given 
training  situation  would  actually  defeat  the  purpose  of  this  report, 
because  the  application  of  learning  principles  must  vary  according  to 
training  programs  and  according  to  day-to-day  instructional  problems. 
The  purpose  here  is  to  help  personnel  responsible  for  the  utilization  of 
ATDs  derive  sound  instructional  practices  and  approaches  to  solutions  of 
training  problems  as  they  evolve.  Judgments  concerning  the  application 
of  these  principles  will  always  be  necessary,  and  the  discussions  which 
follow  are  intended  to  provide  a  foundation  for  structuring  such 
judgments.  It  is  believed  that  these  discussions  will  enable  personnel 
who  are  concerned  with  the  effectiveness  of  ATD  training  to  make  better 
use  of  learning  principles  when  they  train  aircrews. 


TRANSFER  AND  ATD  TRAINING 

The  ultimate  value  of  ATD  training  depends  on  the  extent  to  which 
skills  learned  during  that  training  can  be  utilized  later  In  an  aircraft. 
Skills  learned  in  ATDs  must  either  be  essentially  the  same  as  those 


needed  to  perform  In  aircraft,  or  else  they  must  enable  a  student  to 
learn  other  or  additional  skills  in  aircraft  more  quickly  than  he  could 
if  he  had  not  received  ATD  training.  The  process  of  subsequently  using 
ATO  skills  In  an  aircraft  is  usually  called  transfer  of  training,  or 
simply  transfer.  That  is,  skills  learned  during  ATD  training  are  said  to 
"transfer"  to  aircraft. 

Transfer  Is  a  complex  process.  It  depends  not  only  on  what  is 
learned,  but  on  how  and  under  what  conditions  the  learning  occurs.  The 
complexity  of  transfer  did  not  appear  to  5e  understood  by  personnel 
responsible  for  many  of  the  ATD  training  activities  observed  during  the 
present  project.  Instead,  transfer  was  viewed  as  a  simple,  almost 
mechanical  process.  According  to  this  view,  transfer  will  occur  only 
when:  (1)  the  same  stimuli  are  provided  in  the  ATD  that  are  provided  in 
aircraft;  (2)  a  student  is  able  to  do  the  same  things  with  the  ATD's 
controls  that  he  would  do  with  the  controls  in  aircraft;  and  (3)  the 
feedback  he  receives  from  operation  of  those  controls  is  the  same  as  It 
would  be  In  the  aircraft.  Following  these  assumptions,  the  devices  are 
often  used  to  teach  only  those  skills  that  can  be  performed  in  them  the 
same  way  they  are  performed  in  aircraft. 

Given  sufficient  stimulus  and  task  fidelity,  ATD  performance  will 
transfer  to  an  aircraft.  However,  reliance  only  on  stimulus  and  response 
fidelity  is  a  restrictive  approach  to  ATD  design  and  utilization  because 
It  Ignores  some  of  the  most  important  aspects  of  transferable  skill 
learning.  At  the  same  time,  this  approach  usually  results  in  expensive 
capital  outlays  for  high  fidelity  devices  when  in  some  cases  high 
fidelity  Is  not  necessary. 

To  provide  a  foundation  for  a  more  comprehensive  approach  to  ATD 
training.  It  is  necessary  to  examine  the  concept  of  transfer  Itself. 
Three  factors  that  affect  transfer  will  be  discussed:  (1)  cue  develop¬ 
ment;  (2)  cue  and  response  discrimination;  and  (3)  generalization.  These 
factors  will  then  be  related  to  what  has  been  called  "psychological 
fidelity,"  a  concept  basic  to  transfer  of  ATD  training  to  aircraft. 

The  three  factors  discussed  are  closely  interrelated.  In  fact, 
together  they  constitute  the  basic  foundations  of  aircrew  skill 
performances.  They  will  be  considered  separately  in  this  discussion, 
however,  in  order  to  emphasize  the  implications  of  each  for  ATD  training. 
Implications  that  would  be  less  evident  if  they  were  all  lumped  together. 
(Chapter  IV  develops  the  implications  of  these  factors  as  they  apply  to 
teaching  procedures  and  conditions  for  ATD  practice.) 

Cue  Development 

Aircrews  depend  on  cues  to  initiate  actions,  to  guide  their 
performance  of  those  actions,  and  to  signal  when  an  action  should  be 
altered  or  ended.  The  first  step  In  examining  transfer,  then,  is  to 
define  "cue"  and  distinguish  between  a  cue  and  a  stimulus.  Stimuli  are 
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the  bases  for  cues,  but  a  stimulus  is  not  a  cue  by  itself.  The  term 
"stimulus"  will  refer  only  to  a  physical  object  or  event  that  can 
activate  a  sense  receptor.  For  example,  when  flying  under  turbulent 
conditions,  g  forces  on  the  body  stimulate  internal  receptors  in  the 
muscles,  and  instrument  fluctuations  stimulate  the  eye.  The  training 
task  of  the  aircrewman  is  to  learn  the  meaning  of  these  stimuli,  to 
derive  pertinent  information  from  them,  so  that  the  proper  response  can 
be  made. 

As  these  meanings  are  learned,  stimuli  become  cues.  In  other  words, 
a  cue  is  a  stimulus  that  has  acquired  meaning,  i.e.,  it  conveys 
Information  that  is  understood  by  the  aircrewman.  A  goal  of  aircrew 
training  is  to  learn  the  information  content  of  task  relevant  stimuli  so 
that  precise  actions  can  be  taken.  Even  a  "simple"  automatic  action  such 
as  adjusting  input  to  the  flight  controls  to  counteract  the  effects  of 
turbulence  is  in  reality  a  complex  process  of  using  the  information 
provided  by  motion  and  other  cues.  The  effects  of  g  forces  on  parts  of 
the  body,  the  subtle  variations  in  pressures,  the  directions  and 
magnitudes  of  instrument  changes,  etc.,  all  must  be  "translated"  into 
requirements  for  immediate,  highly  precise  control  inputs. 

Such  processes  are  complex  and  are  often  difficult  to  describe.  For 
example,  in  maintaining  position  while  receiving  fuel  from  a  tanker,  the 
pilot  knows  that  director  lights  indicate  whether  his  vertical  and 
fore/aft  positions  relative  to  the  tanker  are  correct,  and  if  not,  the 
direction  of  any  error.  The  lights  (as  well  as  other  cues)  may  tell  him, 
say,  that  he  should  be  farther  aft.  However,  he  cannot  even  tell  himself 
how  much  to  ease  off  on  the  throttle.  This  latter  knowledge  is  mostly  a 
matter  of  "feel,"  i.e.,  interpretations  of  pressure  and  kinesthetic 
(movement)  cues.  While  making  the  throttle  adjustment,  the  pilot  must 
continually  monitor  the  lights  so  as  to  anticipate  when,  and  how  much, 
the  throttle  should  be  eased  forward,  thus  avoiding  an  over-correction. 
The  "feel"  in  moving  the  throttle  is  the  cue  that  determines  the 
adjustment  he  makes,  which  in  turn  is  confirmed  by  information  (cues) 
from  the  director  lights  that  indicate  changes  in  relative  position. 

The  pilot's  responses  in  this  situation  are  not  simply  mechanical 
reactions  to  stimuli.  They  arise  from  interpretations  of  many  cues  and 
knowledge  of  the  performance  requirements  indicated  by  those  cues.  In 
addition  to  visual  cues,  pressures  and  kinesthetic  sensations  from  the 
flight  controls  provide  further  cues  (information).  These  cues  occur 
both  during  and  after  each  control  movement  by  the  pilot.  Each  cue  must 
be  integrated  with  all  others  if  skilled  performance  Is  to  result. 

Being  able  to  interpret  cues  and  to  respond  to  them  appropriately  is 
the  essence  of  aircrew  skills.  This  role  of  cues  as  opposed  to  stimuli 
has  a  major  implication  for  ATO  use.  The  implication  is  that  cue 
Information  available  in  a  device,  rather  than  stimulus  fidelity  per  se, 
should  be  the  criterion  for  deciding  what  skills  or  skill  components  are 
to  be  taught  in  a  particular  device.  Even  though  some  primary  cues  used 
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fn  the  aircraft  to  perform  a  task  may  not  be  available  in  an  ATD, 

examination  may  show  that  sufficient  alternative  cues  are  available. 
Nonvisual  ATDs,  for  example,  have  been  used  to  train  some  final  approach 
and  landing  skills  by  teaching  pilots  to  rely  on  cockpit  instruments. 
The  cockpit  instruments  can  provide  the  alignment  cues  normally  taken 

from  outside  visual  references.  The  pilots  can  use,  for  Instance,  the 
Instrument  Landing  System  (ILS)  to  establish  runway  alignment, 

glideslope,  and  distance  to  touchdown.  Performance  instruments  must  be 
crosschecked  for  maintaining  airspeed  and  rate  of  descent,  just  as  they 
would  be  in  the  aircraft.  The  approach  can  be  flown  to  touchdown. 

Transfer  can  be  expected  in  terms  of  both  the  instrument  crosscheck 
and  the  manipulation  of  flight  controls  during  the  approach  and  landing. 

Cue  and  Response  Discriminations 

Skilled  aircrew  performance  Is  dependent  upon  making  appropriate 

responses  to  cues.  Therefore,  the  two  most  important  considerations  in 
training  are  how  one  learns  to  interpret  cues  and  how  he  selects  the 
correct  responses  to  be  made  to  those  cues.  If  personnel  responsible  for 
the  design  of  ATD  training  programs  have  this  information,  teaching 
procedures  and  conditions  can  be  designed  to  optimize  the  learning  of 
aircrew  skills  In  these  devices. 

Interpreting  stimuli  and  selecting  appropriate  responses  Involve  a 
process  called  discrimination.  Discrimination  Is  simply  the  recognition 
that  a  given  stimulus  or  response  is  different  from  another  stimulus  or 
response.  The  appearance  of  the  horizon  at  the  top  of  a  canopy,  for 
example,  can  be  discriminated  (means  something  different)  from  Its 
disappearance  below  the  line  of  the  instrument  panel.  If  level  flight  is 
desirea,  given  either  of  these  cues,  the  corrective  movement  of  the  stick 
would  be  backward  for  the  first  condition  but  forward  for  the  second. 

The  simplicity  of  this  definition  of  discrimination  should  not 
suggest  that  discriminations  are  simple  processes,  or  that  they  can  be 
easily  learned.  The  more  complex  the  skill  Involved  in  aircrew 
performance,  the  larger  the  number  of  moment- to-moment,  even  instant-to- 
Instant  discriminations  that  must  be  made.  Also,  as  task  complexity 
Increases,  discriminations  depend  upon  very  subtle  differences  in 
patterns  of  numerous  stimuli.  A  simple  maneuver  such  as  a  bank  turn,  for 
example,  becomes  an  increasingly  complex  task  when  it  must  be  made  to 
precise  performance  standards  involving  a  specified  degree  of  bank,  g 
forces,  and  altitude  maintenance.  Ground  and  Instrument  cues  must  be 
Interpreted  in  greater  detail,  and  g  forces  must  be  "read"  more 
specifically.  Further,  control  Inputs  must  be  precisely  coordinated  with 
the  constantly  changing  cues  so  that  deviations  from  the  desired  path  can 
be  avoided.  The  difference  between  a  novice  and  an  expert  when 
performing  such  complex  task^  is  that  the  expert  has  learned  to  derive 
more  detailed  Information  from  the  cues.  He  can  discriminate  subtle 
differences  that  a  novice  cannot.  He  can  also  translate  the  subtle 
meanings  Into  equally  subtle  control  inputs. 


What  the  student  is  doing  as  he  progresses  from  his  first  attempt  to 
perform  a  bank  turn  to  the  time  that  he  can  perform  this  maneuver  with 
precision  is  learning  to  discriminate  the  finer,  more  subtle  cues  and  cue 
patterns  that  are  used  by  the  expert,  and  to  make  more  rrecise  responses 
to  them.  The  speed  with  which  he  learns  is  itself  an  indication  of  how 
difficult  the  discriminations  are  to  learn.  He  can  learn  very  quickly, 
in  fact,  he  needs  only  to  be  told,  that  pushing  the  stick  laterally  rolls 
the  plane.  However,  mastering  a  steep  bank  instrument  turn  requires  use 
of  a  host  of  constantly  changing  visual  cues  as  well  as  pressure  and 
"feel"  cues  arising  from  moving  the  controls  to  the  necessary  positions 
for  maintaining  bank  angle,  airspeed,  and  altitude. 

The  difficulties  involved  in  learning  to  make  di scr imi nations  can  be 
reduced  by  judiciously  controlling  the  sequences  in  which  skills  are  to 
be  learned.  It  is  for  this  reason  that  attempts  have  been  made  to 
identify  aircrew  training  skills  hierarchies.  The  purpose  of  developing 
skills  hierarchies  is  to  determine  which  skills  underlie,  or  are 
components  of,  skills  higher  in  the  hierarchy  so  that  learning  the  higher 
ones  can  be  made  easier  by  building  upon  previously  mastered  skills  that 
are  lower  in  the  heirarchy.  For  example,  once  a  pilot  has  learned  a 
basic  loop,  he  can  use  the  first  part  of  this  maneuver  in  performing  an 
immelmann  turn.  Learning  the  latter  is  made  easier  because  the  first 
half  of  the  turn,  so  to  speak,  has  ;I.°ady  been  mastered. 

The  same  strategy  can  apply  to  learning  discriminations.  Complex 
discriminations  can  be  built  on  separately  learned  basu  discriminations. 
For  example,  formation  flying  requires  mastery  of  complex  distance  cues, 
precise  inputs  to  flight  controls  corresponding  to  subtle  variations  in 
those  distance  cues,  and  rapid  reactions  to  the  relative  movements  of 
another  aircraft.  While  a  student  is  not  likely  to  master  formation 
flying  solely  by  learning  such  discriminations  and  reactions  in  isclation 
from  each  other,  the  overall  formation  flight  task  can  be  learne  more 
easily  if  the  student  has  previously  mastered  distance  estimation  t^sks 
and  the  coordination  of  control  inputs  in  response  to  visual  cues. 
Furthermore,  once  the  student  has  learned  the  distance  estimation  skills 
necessary  for  formation  flight,  these  di scrimi nations  can  be  used  to 
guide  his  inputs  to  flight  controls  when  he  is  approaching  a  tanker  for 
refueling,  or  even  when  closing  on  a  target  for  weapons  release. 

ATDs  have  a  distinct  advantage  over  aircraft  in  aircrew  training  in 
this  respect.  This  advantage  derives  from  the  fact  that  ATDs  are 
adaptable  to  teaching  cues  and  response  discriminations  at  the  time  they 
can  be  most  effective  for  learning  other,  more  complex  discriminations 
and  skills.  Cues  and  responses  cannot  be  broken  down  during  flight  in 
the  same  ways  they  can  be  in  ATDs.  For  safety  reasons,  basic  aircraft 
control  skills  must  be  mastered  before  most  other  skills  can  even  be 
attempted.  In  the  ATD,  the  sequence  of  individual  skills  can  be  taught 
according  to  instructional  efficiency  rather  than  safety  requirements. 
Discriminations  underlying  skills  can  be  taught  in  ATDs  at  times  and  in 
ways  that  promote  the  most  efficient  development  of  skills. 


Any  number  of  previously  learned  discriminations  can  be  useful  in 
learning  new  discriminations.  An  experienced  pilot  can  transition  to  an 
unfamiliar  aircraft  more  easily  than  can  an  inexperienced  pilot  because 
he  has  previously  learned  to  discriminate  among  many  cues  and  response 
requirements  that  are  similar  to  those  encountered  in  the  new  aircraft. 
However,  previously  learned  discriminations  vary  in  the  extent  to  which 
they  can  help  in  learning  new  ones.  "Feel"  cues  learned  in  an  AID  of  low 
dynamic  fidelity,  for  example,  are  not  as  helpful  in  learning  to  fly  an 
aircraft  as  are  cues  learned  in  a  device  with  high  dynamic  fidelity.  In 
fact,  the  cue  di scr iminations  learned  in  a  poorly  designed  device  may 
actually  result  in  negative  transfer,  i.e.,  they  may  interfere  with 
performance  in  the  aircraft.  Thus  the  experience  level  of  pilots 
undergoing  continuation  training  may  enable  them  to  benefit  from  use  of 
low  fidelity  devices,  but  if  those  devices  involve  inappropriate  or 
erroneous  cues  and  responses,  the  pilots  might  acquire  skills  that  would 
have  to  be  "unlearned"  in  order  to  fly  the  aircraft,  or  at  least  it  may 
be  necessary  to  learn  to  ignore  certain  aspects  of  them. 

General ization 


"Generalization"  refers  to  the  use  of  previously  learned  skills  in 
situations  that  are  different  from  the  situations  in  which  they  were 
learned.  Generalizations  are  so  obvious  that  the  need  to  consider  them 
in  training  is  often  overlooked.  Without  general i zation,  our  day-to-aay 
experiences  would  be  without  meaning  or  context.  We  would  have  to  learn 
anew  to  cope  with  every  experience  we  encounter.  Competence  in  aircrew 
skills  would  not  carry  over  from  one  situation  to  another,  or  even  from 
one  day  to  the  next.  In  fact,  much  of  what  is  usually  termed  "memory"  or 
"retention"  is  really  an  example  of  the  generalization  of  prior  learning 
to  subsequent  situations. 

Generalization  occurs  to  the  extent  that  a  given  situation  is  inter¬ 
preted  as  similar  to  a  previously  experienced  situation.  Similarity  is 
thus  based  on  cue  information,  i.e.,  the  meanings  conveyed  by  the  stimuli 
present  in  the  two  situations.  Procedures  learned  in  a  low  fidelity 
cockpit  mock-up  can  be  generalized  to  (i.e.,  performed  in)  a  high 
fidelity  simulator  or  an  actual  aircraft  because  although  the  two  may 
differ  as  actual  stimuli, the  meanings  of  the  cues  present  in  the  mock-up 
are  similar  to  the  meanings  of  corresponding  cues  present  in  the 
aircraft.  Pretending  to  perform  a  complex  procedural  task  by  pretending 
to  push  buttons  and  position  switches  in  a  mock-up  where  the  buttons  and 
switches  are  represented  symbolically  can  have  the  same  meaning  (and 
training  value)  as  actually  performing  these  actions  in  the  aircraft. 

Thus,  all  cues  Teamed  in  an  ATD  can  be  generalized  to,  i.e., 
subsequently  utilized  in,  aircraft  to  the  extent  that  the  cues  have  the 
same  meanings  in  both  vehicles.  The  physical  stimuli  can  vary. 
Instruments  can  be  of  different  sizes  or  configurations;  visual  displays 
can  resemble  geometric  patterns  more  than  real  world  scenery;  platform 
motion  systems  can  (as  they  must)  be  restricted  to  accelerations  of  brief 
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durations  and  movements  of  small  distances.  Even  cues  associated  with 
the  training  process  itself  can  vary.  Flights  can  be  interrupted  with 
periods  of  freeze  or  playback;  a  landing  approach  may  be  practiced 
repeatedly  without  ever  following  through  the  complete  task  of  first 
flying  to  the  point  to  begin  the  approach;  a  single  instructor  can 

provide  voice  simulation  of  multiple  ground  stations  and  other  aircraft. 
These  and  a  host  of  other  "unrealistic"  or  "low  fidelity"  conditions  can 
be  imposed  in  order  to  achieve  ATD  training  at  reduced  costs,  provided 

the  similarity  of  meanings  of  cues  in  the  devices  and  in  aircraft  can  oe 

maintained. 

The  fact  must  be  remembered  that  even  the  most  sophisticated  ATD 
provides  at  best  a  low  fidelity  representation  of  the  real  world  mission 
of  the  aircraft  simulated.  Therefore,  it  is  important  that  ATD  training 
focus  on  general izable  meanings  of  the  cues  they  can  provide.  In  other 
words,  insofar  as  device  features  permit,  a  student  should  learn  to  use 
cues  that  he  will  need  in  order  to  perform  in  the  aircraft  by 

concentrating  on  the  meanings  of  stimuli  available  in  the  ATD  rather  than 
upon  the  physical  characteristics  of  those  stimuli.  Some  currently 
existing  simulated  ground  visual  scenes,  for  example,  are  composed  of 
nothing  more  than  a  checkerboard  pattern  of  light  and  dark  areas.  Yet, 
as  Stark  (1976)  reported,  experienced. .fighter  pilots  flying  the  Simulator 
for  Air-to-Air  Combat  (SAAC)  used  such  scenes  effectively  as  cues  to 
direction  and  speed  of  movement  while  performing  combat  maneuvers.  If 
pilots  can  use  such  checkerboard  patterns  as  cues  to  the  performance  of 
tasks  requiring  actual  ground  scenes  when  performed  in  flight,  there  must 
be  common  meanings  between  the  two  kinds  of  stimuli.  Although  cue  values 
for  an  experienced  pilot  and  for  a  student  in  training  may  differ,  the 
student  could  almost  surely  learn  some  cue  Interpretations  from  the 
simulated  scenes  that  he  would  be  able  to  apply  to  the  cues  encountered 
during  subsequent  training  in  aircraft. 

Teaching  general izable  cue  interpretations  is  stressed  in  Chapter 
IY.  It  Is  also  emphasized  there  and  in  Chapter  V  that  a  student  will 
need  to  practice  in  the  aircraft  using  cues  learned  in  ATDs,  for  only 
through  such  practice  will  he  learn  to  discriminate  cues  that  are  common 
to  the  ATD  and  the  aircraft  from  those  that  are  not.  With  little  or  no 
experience  in  flight,  for  example,  it  is  likely  that  a  student  would 
probably  use  some  ATD  stimuli  peculiar  to  the  checkerboard  nature  of  the 
visual  display  and  perhaps  miss  the  important  cues  available  from  such  a 
display.  Therefore,  to  ensure  generalization  of  appropriate  cues,  he 
will  need  some  experience  in  aircraft  as  an  important  part  of  the 
learning  process. 

The  learning  discussed  in  the  paragraph  above  might  be  described  as 
generalizing  cue  discriminations,  a  process  that  underlies  all  skill 
learning.  This  process  was  implied  in  the  earlier  discussion  of 
hierarchies,  i.e.,  building  new  discriminations  on  those  already  learned. 
Cues  and  responses  associated  with  previously  learned  skills  become 
incorporated  in  new  skills  through  this  process.  The  first  half  of  a 
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loop  Is  used  to  initiate  an  Immelmann  turn  and  a  Cuban-eight.  The 
sequence  of  thought,  perceptual,  and  kinesthetic  cues  previously  learned 
in  association  with  the  loop,  and  the  related  sequence  of  coordinated 
responses,  are  generalized  to  become  components  of  different  tasks.  The 
Immelmann  and  Cuban-eight  are  not  entirely  new  tasks.  They  build  on  one 
that  is  already  familiar. 

A  major  difference  between  training  as  it  must  occur  in  an 
aircraft,  and  as  it  can  occur  in  an  ATD,  is  the  extent  to  which  training 
devices  permit  concentrated  practice  on  smaller  and  more  specific 
components  of  skills.  Executing  a  power-off  stall  cannot  be  done 
piecemeal  in  an  aircraft.  On  the  other  hand,  if  a  student  has  trouble 
decelerating  at  the  proper  rate  or  not  over-reacting  to  a  "mushy"  stick, 
ATDs  with  re-initial ization  capabilities  can  permit  intensive  practice  on 
the  troublesome  cue-response  discriminations  without  the  time  or  cost 
involved  in  completing  the  entire  task  during  each  trial  as  would  be 

required  in  an  aircraft.  Then,  a  given  degree  of  stick  "mushiness"  can 
guide  the  learning  of  power  control,  or  vice  versa.  Or,  as  in  the 
earlier  example  of  learning  to  discriminate  distance  cues,  changes  in  a 
broad  range  of  distances  can  be  presented  in  concentrated  ATD  practice 
sessions.  The  visual  discriminations  learned  through  such  concentrated 
practice  can  then  be  generalized  as  needed  for  formation  flight, 

approaching  a  tanker,  or  closing  on  a  target.  These  more  complete, 
operational  tasks  could  then  be  practiced,  either  in  an  aircraft  or  an 
ATD,  with  minimum  effort  devoted  to  learning  to  discriminate  crucial 
visual  cues. 

Generalization  of  the  discrimination  among  cues  is  the  basis  for  the 
generalization  of  responses  appropriate  to  those  cues.  If  a  response  has 
become  associated  with  a  cue,  and  refined  and  honed  according  to  the 
requirements  dictated  by  the  cue  in  all  its  subtle  aspects,  then  it  will 
be  available  to  the  extent  the  cue  itself  is  recognized  in  a  subsequent 
situation.  The  challenge  in  ATD  training  is  to  teach  cue  discriminations 
in  such  a  way  that  cue  subtleties  remain  clearly  recognizable  in  the  new, 

different  stimulus  complex  presented  in  an  aircraft.  When  ATD  training 

accomplishes  this,  aircrews  will  transfer  the  associated  responses  to  the 
aircraft  as  wel 1 . 

Sometimes  even  apparently  well-learned  ATD-based  skills  are 
performed  incorrectly  when  first  practiced  in  an  aircraft.  Such 
disruptions  are  not  due  so  much  to  an  inability  to  generalize  responses 
learned  in  the  device  to  the  aircraft  as  to  the  interfering  effects  of 
new  aircraft  cues  which  did  not  exist  in  the  ATD.  The  new  cues  call 
forth  different,  and  possibly  even  competing,  responses.  Anxiety,  for 
example,  introduces  a  host  of  internal  reactions  that  cue  responses 
different  from  those  required  for  precise  performance.  The  extraneous 
responses  interfere  with  the  desired  ones,  so  it  may  appear  that  ATD 
training  was  of  limited  value.  However,  when  the  new  cues  are 
accommodated,  l.e.,  when  they  can  either  be  ignored  by  the  student  or 
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assimilated  as  extensions  of  ATD  cues,  the  ATD-learned  responses  can 
emerge  with  their  original  precision. 

Adaptation  by  pilots  to  aircraft  motion  during  an  intricate  maneuver 
illustrates  this  point.  Sustained  motion  cannot  be  experienced  in 
currently  available  ATDs.  Thus,  no  matter  how  well  aircraft  control 
responses  are  mastered  beforehand  in  an  ATD,  the  actual  continuing  motion 
of  an  aircraft  will  introduce  stimuli  that  were  not  present  when  the 
responses  were  being  learned.  If  a  pilot  associates  a  given  control 
input  with  the  cessation  of  g  forces  resulting  from  the  limited  movement 
necessary  in  an  ATD,  the  sustained  movement  of  an  aircraft  may  well  be 
interpreted  as  not  enough  (or  too  much)  control  input.  In  attempting 
corrections,  precision  of  control  will  suffer  a  decrement.  However,  once 
the  pilot  adjusts  to  the  additional  motion  stimuli  found  in  the  aircraft, 
the  responses  he  learned  in  the  ATD  will  not  be  disrupted.  The  ease  with 
which  those  responses  can  be  incorporated  into  his  performance  in  the 
aircraft  will  be  determined  largely  by  the  extent  to  which  he  mastered 
instrument  or  visual  cues  and  cue-response  relations  in  the  ATD.  If  ATD- 
learned  discriminations  are  of  the  right  sort,  and  are  sufficiently 
mastered,  the  meaning  of  motion  or  other  stimuli  can  be  learned  by 
relating  them  to  the  cues  that  are  already  known. 

Two  examples  will  illustrate  further  the  role  of  generalization  in 
ATD  training.  During  a  visit  to  a  training  site,  an  ATD  instructor 
commented,  "Our  simulator  students  don't  do  as  well  on  navigation  during 
checkrides  as  the  ones  that  were  trained  in  the  aircraft,  but  we  think 
we're  turning  out  better  navigators."  He  was  probably  correct.  Students 
trained  only  in  the  aircraft  had  very  few  places  to  land--the  same  place 
where  they  took  off  and  a  few  nearby  airports.  They  received  intensive 
practice  using  instruments  and  other  navigational  aids  to  approach  these 
few  airports.  On  the  other  hand,  students  in  the  simulator  practiced  a 
larger  number  of  different  simulated  approach  problems,  each  involving 
the  use  of  different  navigational  information.  Due  to  restrictions  on 
training  time,  however,  no  one  simulated  approach  could  be  mastered  to 
the  extent  permitted  by  practice  of  the  ''home  field"  approach  in  the 
aircraft. 

The  instructor  quoted  above  was  saying,  in  effect,  that  if  an 
approach  problem  were  presented  that  had  not  been  practiced  by  either 
group,  the  simulator  students  would  be  superior  in  their  performance. 
There  are  excellent  reasons  for  accepting  the  instructor's  subjective 
impressions,  assuming  the  training  was  properly  conducted.  The  implied 
objective  for  all  students  was  to  be  able  to  respond  to  appropriate  cues 
when  approaching  any  airport.  Simulator  students  practiced  a  large 
varety  of  such  tasks.  Thus,  they  practiced  applications  of  their 
knowledge  of  navigational  procedures  to  different  cue  complexes,  i.e., 
they  learned  to  discriminate  among  a  greater  variety  of  navigational  and 
approach  cues  and  thus  were  better  prepared  to  generalize  their  skills  to 
new  situations.  In  effect,  they  were  practicing  the  process  identified 
here  as  generalization.  It  is  axiomatic  that  a  person  learns  to  do 


whatever  he  practices  during  training.  If  generalization  is  practiced, 
generalization  will  be  learned. 

The  ai rcraft-only  trainees  also  got  some  practice  in  making  discrim¬ 
inations  and  generalizations,  although  their  approaches  were  mostly  to  a 
single  landing  field.  Even  at  that  field,  they  undoubtedly  experienced 
some  variations  in  conditions  from  approach  to  approach.  Even  so,  it  is 
unlikely  that  these  pilots  achieved  the  general  operational  competence 
required  to  land  at  a  variety  of  airfields  as  early  as  those  whose 
practice  involved  a  greater  runge  of  cue-response  generalizations  during 
ATD  training. 

At  another  training  site,  a  T-40  device  was  used  by  KC-135  and  B-52 
pilots  for  practice  and  training  in  instrument  flight.  The  T-40  is  an 
instrument  trainer  that  was  not  designed  to  simulate  either  of  these 
aircraft.  What  this  device  provided  were  opportunities  to  practice  a 
variety  of  instrument  flight  and  navigational  tasks  and  to  learn  to  make 
the  discriminations  required  by  those  tasks.  The  practice  concentrated 
on  cue  and  response  coordinations  that  could  be  generalized  regardless  of 
the  physical  features  of  the  device  itself. 

The  training  conducted  in  the  T-40  illustrates  an  important  point 
regarding  the  use  of  ATDs.  Meaningful  practice  of  the  discriminations 
and  generalizations  needed  to  perform  in  some  flight  tasks  does  not 
require  strict  task,  much  less  physical,  fidelity.  What  one  actually 
does  in  the  T-40  cannot  be  identical  to  what  is  done  in  the  KC-135  or 
B-52  simply  because  the  switches  and  controls,  their  dynamics,  and  their 
configurations  are  different.  Practice  can  be  productive  in  such  devices 
nevertheless,  because  the  meanings  of  the  instrument  flight  and 
navigational  cues  present  in  them  compensate  for  lack  of  stimulus  and 
response  fidelity.  In  fact,  the  majority  of  experimental  studies 
demonstrating  transfer  of  skills  learned  in  devices  to  training  or 
operational  aircraft  have  involved  such  devices. 

Trans fer  and  "Psychological  Fidelity " 

The  design  of  many  ATOs  and  their  use  at  some  of  the  units  surveyed 
during  the  present  project  reflect  an  apparent  belief  that  the  stimuli 
and  associated  responses  in  an  ATD  must  be  similar,  or  even  identical,  to 
those  in  the  target  aircraft  if  skills  learned  in  the  device  are  to 
transfer  to  the  aircraft.  Beliefs  such  as  this  are  based  on  "similarity" 
explanations  of  transfer.  These  explanations  hold  that  the  amount  and 
kind  of  transfer  depend  upon  the  degree  of  stimulus  similarity  and  on  the 
degree  to  which  responses  are  similar  or  antagonistic.  Positive,  i.e., 
desirable,  transfer  occurs  when  ATD  and  aircraft  stimuli  and  associated 
responses  are  similar.  On  the  other  hand,  if  stimuli  are  similar  and 
associated  responses  are  different,  negative,  i.e.,  undesirable,  transfer 
would  result.  No  transfer  occurs  when  stimuli  are  totally  dissimilar, 
regardless  of  responses. 
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According  to  similarity  explanations,  maximum  positive  transfer 
occurs  when  stimuli  and  responses  In  the  A7D  are  identical  to  those  in 
the  aircraft.  Thus,  transfer  from  one  F-15  to  another  is  very  high, 
because  all  stimuli  and  responses  would  be  virtually  identical. 
Howe-er,  transfer  from  an  ATD  to  an  F-15  would  depend  on  the  degree  of 
physical  fidelity  of  the  device.  Maximum  negative  transfer  would  occur 
when  stimuli  are  identical  but  responses  are  antagonistic.  For  example, 
if  in  orcier  to  bring  the  nose  up,  a  forward  stick  movement  were  required 
in  a  simulator,  instead  of  a  rearward  movement  as  in  the  aircraft, 
transfer  would  be  negative,  because  the  two  responses  would  be 
antagonistic  in  spite  of  the  physical  similarity  of  the  simulator  stimuli 
to  those  of  the  aircraft.  Also,  some  degree  of  negative  transfer  would 
be  expected  when  ATD  responses  differ  noticeably  in  magnitude,  or  when 
the  ATD  and  aircraft  differ  in  reactions  to  control  inputs. 

Similarity  explanations  of  transfer  notwithstanding,  it  has  been 
established  in  many  transfer  studies  that  device  and  aircraft  stimuli  and 
associated  responses  need  not  be  physically  similar  for  transfer  to  take 
place.  For  example,  Prophet  and  ESoy3~ (1970)  demonstrated  that  cockpit 
procedures  training  in  a  cockpit  mock-up  made  of  plywood,  photographs, 
and  dowel  rods  transferred  to  an  aircraft  as  well  as  did  corresponding 
training  in  a  device  of  high  physical  fidelity.  (Training  in  each  device 
was  comparable  to  corresponding  training  in  the  aircraft  itself!)  To 
account  for  effective  training  with  such  devices,  it  is  necessary  to 
explain  transfer  as  dependent  on  something  more  than  strict  stimulus  and 
response  similarity.  Explanations  must  somehow  involve  the  equivalence 
of  stimulus  and  response  meanings. 

Explanations  based  upon  meaning  emphasize  cues  rather  than  stimuli, 
that  is,  information  provided  by  stimuli  rather  .than  physical 
characteristics  of  the  stimuli  themselves.  According  to  these 
explanations,  transfer  occurs  when  stimuli  and  associated  responses  in  an 
ATD  can  be  interpreted  by  an  aircrewman  as  equivalent  to  corresponding 
stimuli  and  associated  responses  in  the  target  aircraft.  The  term 
"psychological  fidelity"  or  "psychological  realism"  (as  opposed  to 
physica1  fidelity  or  realism)  is  sometimes  used  to  denote  this 
equivalence  in  mining.  A  device  may  be  useful  for  training  in  spite  of 
its  physical  dissimilarities  if  it  has  high  "psychological  fidelity."  In 
other  words,  there  is  a  psychological  process  that  compensates  for  a  lack 
of  stimulus  and  response  fidelity. 

How  this  psychological  process  operates  is  illustrated  in  an  example 
provided  by  Bunker  (1978).  He  asked  experienced  pilots  to  fly  a 
simulator  with  a  new  computer  generated  image  display  system.  These 
pilots  had  difficulty  in  using  simulated  ground  scenes  that  contained 
curved  roads  and  fields  with  a  variety  of  shapes  and  sizes,  two  features 
that  are  characteristic  of  actual  ground  patterns.  When  all  fields  were 
made  square,  and  all  roads  parallel  or  perpendicular,  however,  the  same 
pilots  had  no  difficulty  using  the  simulator  scene.  Citing  research 
literature  on  visual  depth  perception.  Bunker  pointed  out  that  perceived 
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distance  is  determined  by  gradients,  that  is,  by  how  parallel  straight 
lines  appear  to  close  toward  each  other  as  they  are  more  distant,  or  by 
how  apparent  speed  relative  to  ground  objects  changes  with  altitude,  etc. 
In  brief.  Bunker  noted  that  pilots  do  not  key  on  "naturalness"  of  a 
ground  scene,  but  abstract  from  it  certain  characteristics  such  as 
gradients  that  they  have  learned  to  interpret.  Thus,  to  the  pilot,  a 
scene  is  "psychological  1y  real,"  and  he  can  use  it  as  he  would  use  an 
actual  scene,  if  he  can  readily  discriminate  those  characteristics  on 
which  he  has  learned  to  depend.  Apparent  physical  realism  that  does  not 
include  such  cues  cannot  serve  the  same  purpose  for  him. 

The  preceding  discussions  of  cue  development,  cue  and  response  dis¬ 
crimination,  and  general ization  identified  elements  in  the  development  of 
psychological  processes  upon  which  psychological  fidelity  or  realism 
depends.  In  Bunker's  example,  visual  scene  gradients  become  cues  as 
pilots  learn  to  abstract,  i.e.,  discriminate,  them  from  fields  and  roads 
as  such.  A  road  is  not  a  road  as  a  depth  cue.  Rather  it  is  two  lines 
that  appear  to  converge  in  the  distance.  Once  a  pilot  discriminates  the 
gradient  from  the  air,  he  can  readily  generalize  it  to  any  ground  scene 
by  scanning  the  scene  for  this  meaningful  cue. 

This  overall  psychological  process  of  seeking  and  recognizing 
familiar  cues  in  changing  situations  is  an  instance  of  "mediation."  That 
is,  the  generalization  of  previously  learned  discriminations  to  a 
situation  at  hand  is  an  intermediary  process  that  provides  meaning  for 
the  situation.  It  comes  between,  or  mediates,  the  acts  of  sensing  a 
stimulus  and  responding  to  it. 

Mediation  occurs  any  time  a  person  interprets  a  stimulus  and  acts 
according  to  the  interpretation.  In  this  sense,  even  well  habi tual i zed, 
mechanical  actions  of  aircrew  are  mediated.  Minor  control  adjustments  to 
turbulence,  for  example,  are  not  simple  reflex  reactions  originally. 
They  must  be  learned,  and  the  learning  involves  a  complex  processing  of 
information  from  various  stimuli.  Magnitudes  of  g  forces  are  represented 
as  tactile,  muscular,  and  vestibular  stimuli;  extra-cockpit  visual 
references  add  other  stimuli,  as  do  instrument  changes.  These  stimuli 
must  be  interpreted  and  "translated"  into  types  and  magnitudes  of 
required  control  adjustments.  In  turn,  the  control  adjustments  have  to 
be  guided  by  another  complex  of  stimuli,  i.e.,  the  "feel"  of  controls  and 
their  movement.  "Feel"  includes  numerous  pressure  stimuli  in  the  hands 
and  feet  as  well  as  those  arising  from  moving  these  members. 

Mediations  underlie  all  skills,  and  mediational  explanations  of 
transfer  are  important  in  ATD  training  because  they  focus  attention  upon 
related  learning  processes  rather  than  on  the  physical  features  of  the 
device  and  the  responses  that  can  be  performed  in  it.  During  training, 
the  aircrewman  does  not  simply  react  to  stimuli  presented  by  the  device. 
Simply  performing  tasks  in  the  device,  even  an  exceptionally  high 
fidelity  one,  does  not  constitute  efficient  training.  Instead,  ATD 
training  should  emphasize  the  mediating  processes  that  enable  students  to 
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establish  cue  meanings  and  cue  and  response  discriminations,  and  to 
generalise  skills  being  learned  in  the  device  to  the  stimulus  situation 
to  be  encountered  in  the  airplane.  A  high  degree  of  physical  fidelity  in 
the  device  may  or  may  not  be  needed.  The  point  is  that  ignoring  the  role 
of  meiiation  in  ATD  training,  and  depending  solely  upon  physical 

similarity  as  a  basis  for  transfer,  limits  device  training  to  those 

skills  associated  directly  with  the  device's  level  of  physical  fidelity. 

Effective  training  can  be  obtained  in  devices  that,  to  one  degree  or 

another,  are  dissimilar  to  the  aircraft  simulated.  For  example, 

photographs  of  cockpit  switches  in  the  mock-up  built  by  Prophet  and  Boyd 
(1970)  symbolized  the  real  switches'"  in  the  aircraft,  and  the  symbolic 
process  of  pretending  to  reposition  those  switches  was  just  as  useful 
when  learning  to  perform  cockpit  procedures  tasks  and  their  sequences  as 
actually  performing  them  in  the  aircraft.  Although  there  was  little 
physical  stimulus  similarity  between  the  photograph  and  the  switch,  or 
between  pretending  to  move  the  switch  in  sequence  and  moving  it,  the 
psychological  similarity  of  the  acquired  meaning  of  these  stimuli  and 
responses  was  sufficient  for  the  substitution  of  one  for  the  other  during 
training. 

Furthermore,  the  necessity  for  written  materials  and  oral 
instruction  in  aircrew  training  illustrates  the  extensive  dependence  on 
symbolic  meanings.  With  appropriate  prior  experience,  "mental"  or 
imaginative  rehearsal  can  be  effective  for  learning  or  improving 
performance  in  many  tasks.  Following  their  extensive  review  of 
literature  on  learning,  Wheaton,  Rose,  Fingerman,  Korotkin  and  Holding 
(1976)  stated  that  for  some  skills,  mental  rehearsal  could  even  be 
substituted  on  a  one-to-one  basis  for  some  practice  ,  trials.  The 
requirement  for  effective  training  in  ATDs  is  that  during  practice, 
critical  aspects  of  the  operational  conditions  be  represented  in  the 
experience  of  the  learner.  The  representations  themselves  may  be  derived 
from  high  fidelity  stimulus  conditions,  substitute  stimuli  that  have 
acquired  appropriate  meanings,  a  printed  page,  or  even  a  verbal 
instruction  beginning  with  "Let's  pretend..." 

The  importance  of  mediation  in  ATD  learning  becomes  even  more 
obvious  when  one  considers  the  pervasive  role  of  verbal  mediation--a 
process  with  no  objective  fidelity  whatsoever--both  in  training  and  in 
operational  performance.  The  profound  role  of  language  in  learning  to 
discriminate  among  cues  and  to  generalize  t-hose  discriminations  is  easily 
established:  Two  or  three  brief,  cryptic  lines  of  clearance  are 
sufficient  to  direct  an  aircraft  to  a  distant  airport  via  an  almost 
unlimited  number  of  routes.  Such  is  possible  because  the  clearance 
employs  discriminative  languaye  that  has  been  mastered  by  aircrew  and  air 
traffic  control  personnel,  and  they  can  generalize  their  ability  to 
discriminate  the  cues  contained  in  a  clearance  to  almost  any  new  routing 
task  that  confronts  them.  In  short,  they  are  transferring  previously 
learned  responses  to  new  stimuli  because  they  are  able  to  discriminate 
the  essential  information  in  the  clearance. 
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Without  language,  aircrew  training  would  be  a  chaotic  enterprise. 
Consider  training  a  student  without  any  words,  oral  or  written,  to 
explain  things  to  him  or  tell  him  what  to  do  or  how  to  practice,  hot 
only  does  the  instructional  process  itself  depend  upon  previous  mastery 
of  this  mediational  system,  but  the  student  must  learn  new  uses  of  it. 
He  must  use  language  overtly  for  communication  during  task  performance; 
but  probably  more  important,  he  must  employ  language  covertly  to  clarify 
most  of  the  cues  he  will  use,  and  to  guide  hi?  discrimination  of  cues  and 
the  responses  associated  with  them.  Furthermore,  this  last  usage  of 
language  often  increases  as  an  aircrewman  matures.  For  example,  when  a 
fighter  pilot  was  asked  how  he  decided  on  various  courses  of  action,  he 
described  several  circumstances,  the  cues  he  sought,  and  how  he  talked  to 
himself  about  them.  In  attacking  another  aircraft  he  would  say,  "He's  in 
a  2-g  turn  so  his  radius  will  put  him  over  there.  I'll  set  up  an 

Immelmann  turn."  After  several  similar  examples,  he  broke  off  with,  "1 
talk  myself  through  a  lot  of  things." 

Because  of  the  role  verbal  processes  can  play  in  complex 

performance,  such  as  air  combat  maneuvering,  and  because  verbal izations 
can  be  brought  under  conscious  control,  language  is  a  prime  mediational 
vehicle  for  teaching  even  nonverbal  perceptual -motor  skills.  Eventually, 
a  pilot  may  learn  to  use,  say,  only  visual  perceptions  of  the  rate  of 

change  of  a  compass  heading  to  guide  some  particular  control  input.  But, 
rate  of  change  is  vague  at  best  to  a  novice.  He  must  first  read  the 

compass  as  headings  or  degrees--words--while  probably  saying  to  himself, 
"That's  too  much;  that's  about  right."  If  one  teaches  the  novice  how  to 

use  language  as  a  mediational  vehicle  while  in  training,  i.e.,  to  talk  to 

himself  about  what  to  notice  and  what  to  do,  he  will  be  able  to  learn 
more  rapidly  to  discriminate  perceptual  and  kinesthetic  cues  and 
associated  motor  responses.  The  end  result  will  be  a  well  established 
coordinated,  complex  habit  that  eventually  may  require  little  or  no 
verbal  guidance. 

The  importance  of  such  mediational  processes  are  generally 
recognized  among  ATD  instructors,  at  least  intuitively.  However, 
provisions  in  aircrew  training  for  mediation  per  se,  as  opposed  to  overt 

task  performance,  are  not  as  common  as  one  might  expect.  It  is  often 

said  that  a  student  should  be  "encouraged"  to  verbalize  to  himself  what 
he  is  doing  in  performing  a  task,  but  few  instances  were  noted  in  the  ATD 
programs  surveyed  during  the  current  project  in  which  a  trainee  was  told 
what  he  should  say  or  how  to  select  what  to  talk  about.  The 
verbalizations,  of  course,  should  concentrate  upon  selection 
(discrimination)  of  appropriate  cues,  including  ongoing  kinesthetic 
feedback  from  control  movements.  In  other  words,  aircrew  trainees  should 
be  taught  to  verbalize  the  discriminations  that  they  are  learning  to  make 
among  such  cues. 

A  possible  use  of  the  ATD  instructional  feature  Freeze  illustrates  a 
potentially  valuable  training  practice  involving  verbal  mediation.  When 
a  student  goofs  during  a  maneuver,  an  instructor  typically  freezes  the 
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action  and  discusses  what  the  student  did  wrong.  Typically,  in  the 
training  activities  observed  during  the  current  project,  the  focus  in 
such  discussions  was  upon  objective  cues,  the  failure  to  interpret  them, 
and  the  resulting  overt  actions.  For  example,  the  instructor  would  say, 
"On  this  dive  bomb  delivery,  your  airspeed  was  too  high.  You  have  to 
check  your  airspeed  and  adjust  it  in  order  to  have  accurate  bombs." 

To  enhance  the  effectiveness  and  efficiency  of  ATD  training,  equal 
emphasis  should  be  given  to  how  the  student  is  processing  information, 
how  and  why  he  selected  or  failed  to  select  particular  cues,  and  what  he 
should  do  to  improve  the  processing  of  the  information  he  obtained  from 
those  cues.  For  example,  the  instructor  might  say,  "On  this  dive  bomb 
delivery,  you  allowed  yoursel f  to  become  preoccupied  with  positioning  the 
bomb  sight.  You  left  the  airspeed  indicator  out  of  your  cross-check. 
This  time  note  the  airspeed  immediately  after  you  have  established  your 
dive.  Retard  the  throttle  half-way  to  idle  to  maintain  your  delivery 
airspeed.  Use  this  rule:  Assume  you  will  gain  twenty  knots  for  each 
1000  feet  that  you  descend  until  you  retard  the  throttle." 

In  discussions  of  the  use  of  mediation  during  aircrew  training, 
several  instructors  expressed  concern  that  students  would  learn  to  depend 
upon  the  verbal  mediating  for  guidance  and  would  not  develop  skill  in 
performing  the  task  being  aided.  They  were  concerned  that  the  students 
might  be  learning  only  how  to  interpret  situations  and  to  derive  cue 
meanings  but  not  how  to  recognize  and  respond  automatically  to 
potentially  threatening  situations.  There  is  no  such  danger.  Mediation 
is  useful  in  training,  but  if  the  mediation  is  not  necessary  to 

performance  of  the  task  being  learned,  the  verbal  mediating  will 
disappear  after  the  task  is  learned.  This  occurs  because  once  the  task 
is  learned,  the  mediation  is  no  longer  useful,  and  steps  in  skilled 
performance  that  serve  no  purpose  soon  drop  out. 

Mediation  of  all  types  permeate  every  aspect  of  aircrew  training  and 
performance.  Therefore,  instead  of  leaving  mediational  processes  to 

develop  on  their  own,  as  is  often  the  case  among  instructors  who  rely 
heavily  on  ATD  physical  fidelity  as  the  basis  for  learning,  these 
procedures  should  be  specifically  targeted  in  training  objectives.  When 
left  up  to  the  students,  any  one  or  more  of  a  number  of  cues  may  be 

selected  for  an  action,  and  they  may  be  interpreted  according  to  a 

variety  of  different  schemes. 


SUMMARY 

There  are  no  mechanical  procedures  for  the  application  of  learning 
principles  to  ATD  instruction,  because  these  principles  are  conceptual  in 
nature  and  cannot  be  given  as  categorical  rules.  Judgments  concerning 
their  application  are  required.  A  foundation  for  structuring  such 
judgments  is  provided  in  this  chapter  through  examination  of  learning 
processes  and  the  concept  of  transfer. 
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The  ultimate  value  of  ATD  training  depends  on  the  extent  to  which 
skills  learned  during  that  training  transfer  to  the  aircraft.  Three 
factors  affecting  transfer  were  discussed:  (1)  cue  development;  (2)  cue 
and  response  discrimination;  and  (3)  generalization.  Being  able  to 
interpret  cues  and  to  respond  to  them  appropriately  is  the  essence  of 
aircrew  skills.  Thus,  ATD  instruction  should  focus  on  teaching 
aircrewmen  to  learn  cues  (i.e.,  the  meaning  of  physical  stimuli),  and  to 
derive  pertinent  information  from  them  so  that  the  proper  response  can  be 
made.  The  aircrewman  must  learn  to  discriminate,  i.e.,  to  recognize  and 
select,  the  cues  and  responses  appropriate  to  a  given  task.  He  must  then 
generalize  these  to  situations  different  from  the  situations  in  which 
they  were  learned. 

While  the  focus  of  all  aircrew  instruction  is  on  teaching  the 
discriminations  underlying  aircraft  performance,  ATDs  have  a  distinct 
training  advantage  over  aircraft  in  that  discriminations  can  be  taught  in 
ATDs  at  times  and  in  ways  that  promote  the  most  efficient  development  of 
skills.  Furthermore,  it  is  the  cue  information  available  in  an  ATD, 
rather  than  stimulus  fidelity  per  se,  that  should  be  the  criterion  for 
deciding  what  skills  or  skill  components  are  taught  in  it.  Since 
transfer  is  based  on  cue  information,  all  valid  cue  interpretations 
learned  in  an  ATD  can  be  generalized  in  both  vehicles.  Thus,  it  is 

important  that  ATD  training  focus  on  the  general izable  meanings  of  the 
cues  they  can  provide. 

Transfer  depends  on  meanings,  whether  ATD  stimuli  and  responses  are 
realistic  or  not.  "Similarity"  explanations  of  transfer,  upon  which  many 
conceptions  of  ATD  training  value  rely,  focus  only  upon  the  physical 
similarity  of  training  devices  to  aircraft,  and  upon  their  capability  to 
provide  for  task  performance  as  it  is  actually  done  in  aircraft.  When 
ATD  training  objectives  are  determined  by  dependence  on  physical  and  task 
fidelity,  ATDs  cannot  be  used  to  their  full  potential.  A  complex 

psychological  process  called  mediation  permits  performance  in  devices  of 
various  fidelity  levels  to  acquire  meanings  similar  to  the  meanings  of 
corresponding  performance  in  aircraft.  Thus,  through  mediation,  ATD 
training  is  not  limited  to  device-aircraft  physical  similarity.  Through 
mediational  processes  such  as  language  and  mental  rehearsal,  the 

effectiveness  of  all  ATDs  is  enhanced. 

This  chapter  focuses  ATD  training  on  the  mediational  foundations  of 
aircrew  performance.  Chapter  IV,  which  is  concerned  with  training 

practices  and  conditions  that  promote  learning  cue  and  response 
discriminations,  their  retention,  and  their  generalization,  continues 
this  focus.  Chapter  V,  which  builds  both  on  this  chapter  and  Chapter  IV, 
discusses  the  structuring  of  ATD  training  to  assure  that  it  is  effective 
and  efficient. 


CHAPTER  IV 


THE  CONDUCT  OF  ATD  TRAINING 


Aircrew  skills  are  acquired  through  the  learning  processes  described 
in  Chapter  III.  Whether  the  skills  are  acquired  efficiently  depends  upon 
how  the  practice  is  conducted.  This  chapter  provides  information  on  how 
ATD  training  should  be  conducted.  Chapter  V  draws  on  this  information  in 
discussing  the  structure  and  scheduling  of  ATD  training. 

The  acquisition  of  aircrew  skills  through  instruction  and  practice 
in  ATDs  is  a  complex  process.  For  ATD  training  to  be  efficient  and 
effective,  the  learner  must  acquire  certain  knowledge,  his  actions  must 
be  guided,  and  he  must  be  informed  of  the  correctness  of  his  actions. 
These  three  factors--knowl edge  acquisition  (i.e.,  cognitive  training), 
informing  the  learner  about  his  actions  (i.e.,  feedback),  and  guiding  the 
learner  (i.e.,  guidance)--underl ie  the  conduct  of  effective  instruction 
in  ATDs  and  in  all  other  training  media. 


COGNITIVE  TRAINING 

Effective  and  efficient  ATD  training  depends  upon  preparation  for 
learning.  That  preparation  takes  place  in  classrooms,  in  briefings,  in 
discussions,  in  ATDs  and  aircraft,  and  in  a  variety  of  other  formal  and 
informal  settings.  It  consists  of  learning  about  the  cues  and  responses 
that  must  be  discriminated;  about  the  purposes  of  those  discriminations; 
about  the  meanings  of  symbols  that  will  substitute  for  aircraft  cues  and 
responses  in  ATDs;  and  about  the  subtle  differences  between  ATDs  and 
aircraft  that  must  be  noted  before  transferring  to  aircraft.  Much  of 
this  preparation  involves  cognitive  processes,  i.e.,  thoughts,  ideas, 
mental  images  and  concepts  that  are  both  verbal  and  nonverbal . 
Basically,  it  consists  of  the  cognitive  foundations  upon  which  ATD 
training,  and  ultimately,  aircrew  performance,  rest. 

Cognitive  Training  in  Skill  Acquisition 


Cognitive  training  should  be  employed  systematically  during  aircrew 
training  to  provide  a  general  context  for  skill  performance  and  to  aid  in 
learning  particular  cue  and  response  discriminations.  Some  examples  of 
aircrew  training  activities  that  have  made  effective  use  of  cognitive 
training  may  be  found  in  the  aircrew  training  literature.  For  example, 
Williams  (cited  in  Fitts  and  Posner,  1967)  conducted  detailed  discussions 
with  novice  pilots  of  each  maneuver  to  be  practiced  on  a  given  flight. 
The  focus  was  primarily  upon  cue  and  response  discriminations.  The  exact 
sequence  and  relative  timing  of  each  cue  and  response  was  delineated, 
such  as  "When  you  cross  the  fence  at  the  end  of  the  runway,  reduce  your 
power  to  idle."  The  students  undergoing  this  training  soloed  in  an 
average  of  3-1/2  hours  in  an  aircraft  compared  to  10  hours  for  a  control 
group.  Prather  (1972)  used  only  an  audio  tape-recorded  analysis  of  a 
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landing  task  for  similar  training  sessions  with  a  group  of  T-37  students. 
The  instruction  on  the  tape  emphasized  what  to  do,  when,  and  why. 
Following  this  training,  these  students  were  rated  significantly  higher 
on  landing  procedures  and  techniques  than  were  students  who  did  not 
receive  It. 

Cognitive  training  need  not  be  restricted  to  verbal  information. 
For  example.  Smith,  Waters,  and  Edwards  (1975)  prepared  and  successfully 
validated  a  training  "package"  for  the  T-37  overhead  traffic  pattern  that 
employed  slides  and  8mm  motion  pictures  to  teach  students  to  discriminate 
the  visual  cues  associated  with  that  maneuver.  Commercial  airlines  have 
made  considerable  use  of  nonverbal  cognitive  training,  employing  a  wide 
range  of  media  resources  to  prepare  students  to  learn  cue  and  response 
discriminations  during  subsequent  ATD  training  sessions  (Browning, 
Copeland,  Lauber,  Nutter,  and  Scott,  1972;  see  especially  Appendix  A). 

Virtually  all  of  the  aircrew  training  processes  noted  during  the 
current  project  made  extensive  use  of  cognitive  training.  In  academic 
training,  for  example,  students  acquire  a  knowledge  of  the  tasks  they  are 
to  perform.  They  are  briefed  before  flights,  and  considerable  amounts  of 
"homework"  are  completed  priu’"  to  those  briefings.  They  are  encouraged 
to  rehearse  tasks  mentally  ant  to  think  them  through  before  attempting 
them  in  ATDs,  and  again  after  they  have  been  practiced.  Debriefings 
following  training  consist  of  analyses  that  highlight  the  adequate 
aspects  of  performance  as  well  as  problems  to  be  resolved  during 
subsequent  training.  These  training  activities  provide  criteria  for  task 
performance.  Identify  purposes  of  tasks,  provide  contexts  for  their 
performance,  and  provided  bases  for  decisions  to  be  made  during  training 
and  subsequently  during  operational  mission  activities. 

Cognitive  training  plays  a  significant  role  during  ATD  training  as 
well  as  in  preparation  for  It.  The  reminder,  "Think  about  what  you  are 
doing,"  Is  frequently  employed  in  the  devices  to  call  forth  a  conscious 
awareness  of  experiences  during  learning  so  that  cue  and  response 
characteristics  will  stand  out.  In  the  programs  judged  more  effective, 
the  students  are  being  guided  concerning  what  they  are  to  think  about  in 
such  situations.  For  example,  when  4-1/2  g  is  achieved  in  an  F-4  at  the 
beginning  of  an  overhead  maneuver,  instructors  reportedly  have  been 
Instructed  to  tell  students  to  think  "feel"  at  this  g  level.  A  number  of 
Instructors  encourage  students  to  "talk  themselves  through"  the 
performance  of  tasks  they  were  attempting  to  learn,  and  give  helpful 
guidance  as  to  what  to  talk  about,  e.g.,  the  cues  to  which  the  students 
should  respond,  when  and  why.  For  example,  in  learning  when  to  depart 
from  a  standard  loop  to  complete  an  Immelmann,  one  instructor  said  he  had 
his  students  verbalize  the  cues  that  Indicate  the  point  for  change  and 
the  consequence  of  delaying  or  initiating  too  soon  the  actions  required 
to  complete  the  Immelmann. 

In  all  of  the  studies  cited  above  and  in  the  examples  of  uses  of 
cognitive  training  noted  in  the  current  project,  the  key  to  successful 
training  was  the  systematic,  highly  specific  identification  of  cues  and 
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responses  that  the  students  would  encounter  during  subsequent  practice  in 
ATDs  or  aircraft.  The  verbal  and  perceptual  information  the  students 
received  during  such  training  became  a  basis  for  the  students  to  guide 
themselves  in  seeking  and  interpreting  cues  that  they  had  to  learn  to 
di scriminate. 

Cognitive  Training  in  ATDs 

The  value  of  ATD  training  in  general  depends  upon  the  use  of 
cognitive  processes  to  equate  (psychologically)  the  cues  and  responses 
learned  in  ATDs  to  those  occurring  in  aircraft.  ATDs  are  physically 
different  from  aircraft,  and  the  lower  the  fidelity  of  an  ATD,  i.e.,  the 
more  symbolic  the  cues  and  responses  involved  in  its  use,  the  greater  the 
need  for  such  equating  mediation  while  practicing  with  it.  It  is  through 
mediation  that  ATDs  of  all  fidelity  levels,  from  weapons  systems  trainers 
to  paperboard  mock-ups  of  aircraft  cockpits,  can  be  used  effectively  as 
trainers.  In  such  devices,  the  student  performs  symbolic  actions  in 
response  to  symbolic  cues.  By  attending  to  the  meanings  of  these  actions 
and  cues,  instead  of  their  physical  characteristics,  transferable 
discriminations  of  cues  and  responses  can  be  practiced  and  learned,  and 
their  ATD-peculiar  physical  characteristics  can  be  ignored. 

But  regardless  of  the  fidelity  of  an  ATD,  an  awareness  of 

operational  reality  should  be  maintained.  Students  training  in  ATDs 
should  be  thinking  constantly,  not  of  what  they  are  doing  in  ATDs,  but  of 
the  meanings  and  effects  of  their  actions  with  respect  to  performance  in 
the  aircraft.  What  sounds  are  heard  when  an  engine  starts  or  when  the 
throttle  is  pushed  forward?  What  are  the  disorienting  effects  of  a 
Cuban-eight  and  what  cues  must  be  relied  on?  What  information  is  needed 
to  perform  a  bomb-toss  maneuver  and  what  cocknit  instruments  can  provide 
it?  It  is  important  that  academic  instruction  and  pre-ATD  session 
briefings  stress  these  meanings  so  that  the  student  can  attend  to  them 
when  attempting  to  learn  the  skills  involved. 

In  addition  to  explanations  of  performance  requirements, 

preparations  for  ATD  training  sessions  should  anticipate  and  guard 
against  a  student's  relating  what  he  does  during  ATD  practice  only  to  the 
device  itself.  When  cues  or  cue  patterns  are  only  partially  represented 
in  an  ATD,  the  missing  portions  of  cue  complexes  that  are  necessary  to 
performance  in  the  aircraft  should  be  described,  and  their  relations  to 
the  cues  that  are  available  should  be  explained  during  the  briefing.  For 
example,  in  many  maneuvers  in  high  performance  fighters,  g  cues  must  be 
coordinated  with  the  "feel"  of  controls  for  smooth,  accurate  gun-tracking 
performance.  Current  state-of-the-art  ATDs  cannot  duplicate  all  elements 
of  these  cues.  Therefore,  the  student  should  think  about  their 
occurrence  in  the  aircraft  while  practicing  in  the  ATD,  and  he  should 
recall  them  during  subsequent  ATD  sessions  as  often  as  necessary  to 
maintain  an  awareness  that  they  are  part  of  the  cue  structure  he  is 
learning. 
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When  cues  are  only  partially  available  as  is  the  case  with  some  cues 
in  all  present  day  ATDs,  students  should  be  made  aware  of  the  cue 
components  that  are  missing.  For  example,  the  changing  angularity  of 
ground  reference  lines  while  an  aircraft  is  in  motion  can  be  represented 
by  currently  available  computer  generated  visual  simulation  systems, 
while  changes  in  the  blocking  of  views  of  one  ground  object  by  another 
sometimes  cannot.  When  practicing  with  such  a  symbolic  visual  system, 
the  student  should  be  instructed  to  focus  on  angularity  cues  (and  told 
what  they  are),  but  to  keep  in  mind  that  in  actual  flight  he  would  also 
experience  additional  blocking  cues  arising  from  movement  of  the  aircraft 
past  objects. 

For  AID  training  to  transfer  to  aircraft,  the  information  value  of 
cues  in  the  two  must  be  comparable,  and  the  responses  learned  in  the  ATD 
must  be  usable  in  the  aircraft  or  adaptable  to  it.  Since  ATD  and 
aircraft  cues  and  reponses  frequently  are  not  identical,  it  is  important 
that  those  that  are  common  to  both  ATDs  and  aircraft  be  discriminated 
from  those  that  are  unique  to  the  ATD,  since  recognition  of  cue 
similarities  and  differences  are  necessary  for  effective  use  of  ATDs.  An 
important  role  of  cognitive  training  is  to  make  these  discriminations 
obvious.  (Analytic  techniques  for  identifying  common  and  unique  cues  and 
responses  can  be  found  in  Caro,  1970.) 

It  was  noted  during  the  present  project  that  ATD  instructors  do  call 
the  attention  of  students  to  differences  between  ATD  and  aircraft. 
However,  the  instructors'  inclinations  often  are  to  avoid  practicing 
tasks  in  ATDs  where  the  differences  occur  instead  of  dealing  with  the 
differences  as  a  cognitive  discrimination  training  requirement.  Even 
worse,  some  instructors  were  found  to  emphasize  the  inadequacies  of  the 
ATD  and  thereby  communicate  to  students  that  the  practice  session  was  of 
little  importance. 

Several  instructors  interviewed  expressed  concern  that  even  if  the 
student  has  been  instructed  as  to  the  differences  between  a  particular 
device  and  an  aircraft,  he  will  adapt  his  skill  to  the  peculiarities  of 
the  ATD,  and  negative  transfer  to  the  aircraft  will  result.  That  is, 
these  instructors  were  concerned  that  the  skills  a  student  takes  from  a 
low  fidelity  ATD  to  an  aircraft  will  interfere  with  proper  performance  in 
the  aircraft.  This  risk  certainly  exists,  but  it  can  be  minimized  if  the 
student  learns  to  discriminate  the  differences  between  the  ATD  and  the 
aircraft  and  attends  to  those  differences  during  practice  in  the  device. 
With  prior  knowledge  of  differences  to  expect,  he  can  key  quickly  on 
aircraft  characteristics  and  correct  for  the  negative  transfer  while 
profiting  from  the  ATD  learning  that  was  appropriate. 


FEED8ACK 

"Feedback"  refers  to  cues  that  inform  a  learner  as  to  the  results  or 
effects  of  his  actions.  Feedback  has  two  important  roles  during  ATD 
training:  (1)  to  maintain  a  student's  motivation  to  learn;  and  (2)  to 


infonn  the  student  of  the  appropriateness  or  inappropriateness  of  his 
interpretations  of  cues  and  his  responses  to  those  cues. 

To  maintain  motivation,  the  student  should  make  progress  toward 
recognized  and  meaningful  goals  during  each  ATD  training  session,  and  at 
least  part  of  the  progress  must  be  apparent  to  him.  Thus,  the  careful 
use  of  feedback  to  aid  skill  acquisition  will  itself  lead  to  motivation 
to  perform  correctly.  In  addition,  positive  feedback  expressed  as 
approval  from  peers  and  superiors  is  generally  helpful  in  maintaining 
maximum  effort  and  favorable  attitudes  toward  training.  (For  a  further 
discussion  of  the  effects  of  feedback  on  attitude  and  motivation,  see 
Chapter  VII.) 

The  second  role  of  feedback,  to  inform  the  student  as  to  the 
appropriateness  of  his  cue  interpretations  and  actions,  is  essential  to 
the  learning  of  skills.  For  example,  a  student  learning  to  maintain  a 
2-1/2  degree  gli deslope  on  a  visual  approach  for  landing  initially  will 
be  unable  to  discriminate  the  visual  gli deslope  cues.  If  he  receives  no 
information  regarding  the  accuracy  of  his  descent  angle,  his  performance 
is  not  likely  to  improve.  On  the  other  hand,  if  he  is  told  when  his 
descent  is  incorrect  and  the  direction  and  the  amount  of  his  error,  his 
performance  will  improve  steadily. 

Intrinsic  Feedback 


There  are  two  kinds  of  feedback:  intrinsic  and  supplemental. 
Intrinsic  feedback  refers  to  Information  that  occurs  naturally  as  a 
result  of,  or  is  inherent  in,  the  performance  itself.  Almost  all 
situations  involved  in  aircrew  performance  contain  a  variety  of  Inherent 
Information.  The  reaction  of  an  aircraft  or  ATD  to  the  movement  of  a 
control  stick  provides  intrinsic  feedback,  as  do  changes  in  cockpit 
Instruments,  aircraft  noise,  indications  of  weapons  impact,  etc.  By 
observing  such  inherent  indicators,  a  pilot's  actions  can  be  adjusted  and 
"tuned."  Operational  skill  will  be  determined  by  the  extent  to  which  an 
alrcrewman  can  discriminate  intrinsic  cues  in  order  to  obtain  the 
feedback  needed  for  his  continued  control  inputs. 

All  ATDs  provide  intrinsic  feedback.  Any  ATD  feature  designed  to 
provide  the  same  cues  that  are  provided  in  the  aircraft  when  the  same 
control  input  is  made  to  each  will  provide  Intrinsic  feedback.  The 
greater  the  degree  of  physical  similarity  between  the  device  and  the 
aircraft  simulated,  the  more  appropriate  Intrinsic  feedback  it  provides. 

Early  In  training,  students  are  not  able  to  discriminate  many  of  the 
cues  that  provide  intrinsic  feedback,  either  missing  them  entirely, 
misinterpreting  them,  or  being  unable  to  glean  the  required  information 
from  them.  For  example,  students  may  be  unable  to  discriminate  the 
specific  engine  sounds  that  provide  feedback  to  experienced  pilots 
concerning  power  settings  and  rates  of  change.  Eventually,  the  student 
must  use  such  feedback  and  must  learn  to  respond  to  it  appropriately. 
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Until  he  learns  of  its  existence  and  meaning,  however,  it  is  of  no  value 
to  him. 


One  way  to  aid  a  student  in  learning  to  depend  on  intrinsic  feedback 
is  to  augment  it  in  some  manner.  Augmented  feedback  is  intrinsic 
feedback  that  has  been  altered  in  such  a  way  that  its  characteristics  can 
be  more  easily  discriminated  by  a  student.  For  example,  freezing  a 
simulator  at  a  critical  point  in  a  maneuver  provides  an  opportunity  to 
examine  cue  patterns  that  he  must  learn  to  recognize  rapidly.  A  replay 
of  a  recorded  maneuver  in  slow  time  permits  greater  opportunity  for  the 
student  to  note  changes  in  instruments,  instrument  patterns,  and 
simulated  motion  and  visual  feedback  cues.  Or,  repetitive  replay  of 
particular  segments  of  a  recorded  maneuver  makes  possible  numerous 
exposures  to  feedback  cues  that  occurred  only  once  while  the  maneuver  was 
actually  being  performed. 

Supplemental  Feedback 


Teaching  a  student  to  discriminate  intrinsic  feedback  cues  often 
requires  additional,  immediately  obvious,  forms  of  feedback  that  are  not 
themselves  inherent  in  operational  performance.  The  earlier  example  of 
an  instructor  providing  error  information  to  guide  a  student  learning  to 
follow  a  gli deslope  during  a  landing  illustrates  a  common  usage  of  such 
additional  feedback.  Initially  the  student  would  not  be  able  to  discern 
the  Intrinsic  cues  that  could  tell  him  when  he  had  a  correct  rate  of 
descent  and  angle  of  attack,  so  the  instructor  provides  the  information 
himself.  Information  of  this  sort  is  usually  termed  "supplemental 
feedback,"  that  is,  it  is  feedback  provided  to  supplement  the  information 
available  from  intrinsic  feedback. 

An  important  function  of  supplemental  feedback  in  skill  acquisition 
is  to  Inform  the  student  of  the  availability  and  relevance  of  intrinsic 
feedback  that  he  should  look  for  and  learn  to  interpret.  For  example, 
Instructors  often  call  out  to  students  practicing  an  approach  to  a  tanker 
the  distances  they  are  from  the  tanker  while  closing  upon  it.  Knowing 
that  he  is  500  feet,  then  300  feet,  50  feet,  etc.,  identifies  points  at 
which  visual  cues  can  be  discriminated.  At  each  point,  the  student 
should  be  shown  the  cue  characteristics  that  he  can  associate  with  the 
distances  called  out  by  the  instructor.  When  he  has  learned  these  cue- 
distance  associations,  +he  instructor's  verbal  feedback  (supplemental) 
will  no  longer  be  needed.  The  visual  cues  themselves  will  provide  the 
feedback  (Intrinsic)  needed  to  perform  the  tanker  approach.  Thus, 
supplemental  feedback  is  most  important  early  in  training  before  a 
student  learns  to  discriminate  cues  that  provide  intrinsic  feedback. 

Supplemental  feedback  is  also  useful  as  an  aid  in  teaching  response 
discriminations.  While  skilled  operational  performance  depends  on 
Intrinsic  feedback,  it  Is  not  necessary  to  wait  until  the  student  is  able 
to  rely  on  intrinsic  feedback  before  relevant  responses  can  be  practiced. 
While  the  student  Is  hearing  the  Instructor  call  out  “300  feet,”  "100 
feet,"  etc.  as  he  closes  on  a  tanker,  he  is  receiving  feedback  regarding 


the  effects  of  his  control  inputs  and  can  learn  to  adjust  closure  rates 
while  simultaneously  learning  to  interpret  distance  cues.  Similarly,  in 
a  dive  bombing  run,  instructor  comments  concerning  the  rate  of  pipper 
closure  or  the  steepness  of  the  dive  tell  a  student  whether  or  not  his 
control  inputs  are  adequate.  Adjustments  necessary  to  correct  dive 
angles  can  thus  be  discriminated  while  learning  to  interpret  the 
intrinsic  cues  that  signal  needed  adjustments. 

In  such  situations,  suppl ementary  feedback  is  necessary  to  aid  the 
student  in  learning  to  discriminate  cue  meanings  and  control  inputs.  A 
student  cannot  learn  to  recognize  a  subtle  cue  or  select  and  fine-tune  a 
response  unless  subtle  variations  in  cues  and  responses  make  a  meaningful 
difference  to  him.  While  learning,  a  difference  is  often  apparent  to  the 
student  only  through  his  instructor's  comments,  direction,  or  evaluation 
of  his  performance.  The  instructor  must  tell  him  that  a  pitch  angle  or 
rate  of  closure  is  correct  or  incorrect,  and  provide  other  information 
that  defines  conditions  the  student  has  not  yet  learned  to  recognize. 
Such  supplemental  feedback  informs  the  student  that  what  he  is 
experiencing  should  be  interpreted  in  terms  of  the  information  provided 
by  the  instructor.  With  practice,  the  ability  of  the  student  to 
recognize  and  respond  to  intrinsic  feedback  will  increase  to  the  roint 
that  he  can  depend  on  his  own  recognition  of  cue  information,  rather  than 
on  supplemental  feedback  supplied  by  an  instructor. 

Supplemental  feedback  can  be  given  freely  in  ATDs ,  either  under 
computer  control  (e.g.,  via  automatically  initiated  aural  alerts  when 
performance  envelopes  are  exceeded)  or  by  the  instructor.  In  addition, 
supplemental  feedback  can  be  administered  through  use  of  the  device's 
various  instructional  features  (e.g.,  freeze  and  record/playback)  when, 
in  the  judgment  of  the  instructor,  it  can  be  most  effective.. 

Possibilities  for  feedback  can  be  numerous  even  in  ATDs  without 
sophisticated  instructional  features.  Many  part-task  trainers  have 
relatively  simple  computer  controlled  or  manually  operated  mechanisms  for 
feedback.  Even  a  paperboard  mock-up  to  teach  cockpit  procedures  can  be 
an  effective  ATD,  provided  only  that  the  student  can  tell  the  difference 
between  an  action  successfully  completed  and  one  that  is  not.  In  many 
cases,  all  he  would  need  for  feedback  is  the  ability  to  recognize  himself 
whether  or  not  he  pressed  a  button,  set  a  switch,  etc.  From  the 
standpoint  of  effectiveness,  it  is  necessary  only  that  a  sufficient 
context  of  meaning  be  provided  so  that  whatever  feedback  is  presented  can 
guide  or  confirm  the  learner's  discriminations  and  resulting  actions. 

Timing  of  Feedback 


The  timing  of  feedback  represents  a  major  area  of  concern  to  aircrew 
instructors.  Several  instructors  indicated  the  belief  that  "immediate" 
feedback  during  training  is  necessary,  but  expressed  difficulty  In 
providing  It.  Further,  they  recognized  the  value  of  post-training  period 
debriefing  sessions,  but  were  concerned  that  delaying  feedback  until  such 
sessions  was  ill  advised  because  of  the  need  for  rapid,  even  immediate, 
feedback. 
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Such  concerns  can  be  resolved  by  a  simple  principle:  Feedback  at 
any  time  can  be  effective  so  long  as  it  can  be  meaningfully  related  co 
the  specific  cue  and  response  discriminations  of  concern.  Sometimes 
feedback  must  be  immediate.  For  example,  in  the  earlier  illustration  of 
an  instructor  calling  out  distances  as  a  student  learns  to  close  on  a 
tanker,  it  is  important  that  the  feedback  "300  feet"  be  pronounced  at  the 
time  the  student's  aircraft  is  approximately  300  feet  from  the  tanker. 
If  this  is  not  the  case,  the  stimuli  the  student  experiences  at  300  feet 
could  not  readily  be  associated  with  a  known  distance. 

Furthermore,  when  feedback  is  needed  for  the  timely  initiation  of  a 
subsequent  response,  the  response  necessarily  cannot  begin  until  the 
feedback  occurs.  For  example,  when  learning  a  coordinated  skill  sequence 
such  as  connecting  with  a  tanker  to  begin  refueling,  the  effects  of  one 
movement  are  often  the  cue  for  immediately  initiating  the  next,  and  no 
observable  feedback  delay  may  be  tolerable.  Likewise,  control  movements 
requiring  nearly  instantaneous  compensating  adjustments,  such  as 
operation  of  flight  controls  during  formation  flight,  must  have  immediate 
feedback.  Otherwise,  the  pilot  would  be  unable  to  make  the  adjustments 
soon  enough  for  the  task  to  continue,  let  alone  for  learning  to  take 
place. 

On  the  other  hand,  delaying  feedback  for  hours  may  not  have 
noticeably  disruptive  effects  on  learning  some  tasks.  For  example,  a 
pilot's  navigation  skills  can  be  sharpened  when  he  deviates  from  the 
intended  course  during  practice,  and  discovers  only  after  considerable 
time  that  he  is  not  where  he  intended  to  be.  For  learning  to  take  place 
in  spite  of  the  greatly  delayed  knowledge  that  he  erred,  it  is  necessary 
only  that  the  pilot  remember  when  his  errors  occurred  and  what  they  were. 

When  the  relationships  between  cues  and  responses  can  be  clearly 
remembered,  delay  of  feedback  is  not  a  serious  problem.  For  example,  an 
instructor's  statement  during  a  post-ATD  debriefing,  "You  were  slow  in 
adjusting  your  airspeed,"  car  be  effective  feedback  if  the  pilot  can 
remember  the  sequence  of  cues  leading  to  the  ones  he  used  during 
practice.  Summaries  of  performance  and  records  of  events  occurring 
during  training,  such  as  are  provided  by  hardcopy  printouts  of  instructor 
station  displays,  for  example,  aid  the  student  in  remembering  just  what 
happened  at  a  particular  time  and  offer  excellent  data  for  feedback 
during  debriefings. 

If  there  is  reason  to  believe  that  pertinent  cues  and  responses  may 
not  be  clearly  recalled  after  the  session,  feedback  should  be  provided 
sooner.  Instruction  in  ATDs  can  be  more  effective  when  prompt  feedback 
is  required  than  can  instruction  in  an  airplane.  The  Instructor  can 
freeze  an  ATD  whenever  it  is  appropriate  to  do  so  to  provide  feedback. 
Promptness  of  feedback  can  be  especially  important  if  the  bases  for  the 
errors,  i.e.,  misinterpretations  of  cues  and/or  erroneous  responses,  are 
likely  to  become  vague  with  time. 
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Problems  uf  Overuse  and  Underuse  of  Supplemental  Feedback 

Supplemental  feedback  poses  special  difficulties.  If  used  too 
freely,  a  student  may  learn  to  depend  on  it  rather  than  learn  to 
discriminate  the  feedback  cues  that  are  a  natural  part  of  (intrinsic  to) 
the  task  being  learned.  When  supplementary  feedback  substitutes  for 
intrinsic  feedback,  the  student  learns  to  key  on  supplemental  feedback. 
Therefore,  supplemental  feedback  should  be  used  only  to 

direct _ attention  toward  intrinsic  feedback.  To  ensure  that  this  is 

happening,  suppl ementaT ~ feedback  should  eventually  be  withdrawn  so  that 
the  student  can  practice  depending  on  himself.  He  should  be  forced  to 
seek  and  interpret  intrinsic  cues. 

Generally,  this  means  that  supplemental  feedback  is  valuable 
primarily  in  the  early  stages  of  learning  to  make  the  cue  and  response 
discriminations  necessary  to  a  particular  task.  As  learning  progresses, 
it  should  be  withdrawn.  Even  early  in  learning,  the  student  should 
sometimes  be  required  to  practice  without  supplemental  feedback  as  a  way 
of  determining  whether  it  is  still  needed. 

Students  vary  in  the  extent  supplementary  feedback  is  needed.  A 
rule  of  thumb  for  its  use  is  available  to  the  instructor,  however: 
Because  it  is  needed  primarily  to  direct  attention  to  cue  and  response 
discriminations,  it  is  not  needed  to  the  extent  these  discriminations  are 
being  learned  without  it.  Therefore,  if  skill  learning  is  progressing 
steadily,  withhold  supplemental  feedback;  if  not,  provide  it  for  a  while, 
then  withhold  it  and  check  progress  again. 

Too  little  use  of  supplemental  feedback  was  found  to  be  more  of  a 
problem  during  ATD  training  than  was  overuse.  For  example,  some  IPs 
withhold  most  comments  regarding  student  performance  until  a  post-flight 
debriefing.  In  one  instance  in  which  a  student  was  practicing  in  an  ATD 
while  transitioning  to  a  single-seat  fighter,  an  IP  commented  that 
because  the  student  would  have  to  depend  on  himself  in  the  aircraft,  he 
should  do  so  in  the  ATD;  therefore  he  should  not  be  given  supplemental 
feedback  in  the  device--an  extreme  case  of  underuse. 

A  widely  noted  misconception  seemed  to  underlie  training  practices 
of  this  sort.  It  is  that  task  fidelity  and  situational  realism  must  be 
maintained  during  all  aspects  of  training.  Certainly,  these  aspects  of 
the  training  situations  are  important,  particularly  during  advanced 
stages  of  ATD  training  and  when  practicing  operational  tasks  in  aircraft. 
However,  the  training  conditions  such  beliefs  foster  are  incompatible 
with  the  use  of  supplemental  feedback  when  it  is  needed  to  direct  the 
student's  attention  to  cues  essential  to  skill  performance.  The 
implication  of  training  practices  based  on  such  beliefs  is  that  a  novice 
can  teach  himself  what  cues  to  attend  to  and  how  to  respond  to  them  as 
efficiently  as  an  expert  instructor  can.  The  implication  is  unsupported 
with  respect  to  aircrew  training. 
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GUIDANCE 


Guidance  is  the  directing  of  actions  of  a  learner  toward  a  desired 
goal.  The  actions  being  guided  may  be  thought  processes,  physical 
movements,  oral  communications,  the  selection  of  cues,  or  the  processing 
of  cue  information.  Guidance  is  involved  every  time  an  instructor 
comments  to  a  student  about  what  he  should  do,  remember,  or  think  about. 

Properly  used,  guidance  helps  in  the  learning  of  aircrew  skills  in 
two  ways.  First,  it  speeds  learning  when  it  identifies  desirable  cues 
and  responses  that  the  student  cannot  recognize  on  his  own.  Second,  by 
identifying  correct  cues  and  responses,  it  reduces  the  likelihood  that 
inappropriate  cues  will  be  used  and  incorrect  responses  made.  Thus,  it 
helps  prevent  learning  of  erroneous  actions  that  would  eventually  have  to 
b£  unlearned. 

Guidance  in  Early  Training 


Guidance  is  especially  important  in  early  training.  A  novice 
aircrewman  is  confronted  with  many  new  potential  cues.  Some  are  not  even 
noticed  at  first,  and  the  meanings  of  many  others  are  only  vaguely 
perceived.  Furthermore,  the  responses  that  should  be  made  to  particular 
cues  are  largely  unknown  to  a  novice,  and  those  that  are  known  frequently 
cannot  be  coordinated  with  their  cues.  Through  guidance,  students  can  be 
led  to  focus  upon  correct  cues,  their  interpretation,  and  the  types  of 
responses  they  require. 

The  most  common  forms  of  guidance  in  aircrew  training  are  written 
and  oral  instructions.  These  are  the  principle  forms  that  have  been 
available  for  use  throughout  most  of  the  history  of  aviation.  However, 
current  state-of-the-art  ATDs  offer  capabilities  for  guidance  that  can  be 
superior  to  words  for  some  instruction.  For  example,  in  an  ATD  with  a 
capability  for  automated  demonstrations,  many  cues  and  responses  that  the 
student  must  learn  to  discriminate  but  that  are  difficult  to  describe 
verbally  can  be  observed  and  even  experienced  by  the  student.  An 
automated  demonstration  can  illustrate  precisely  (within  the  fidelity 
limits  of  the  device)  what  the  student  must  learn  to  do.  If  the 
demonstration  also  includes  movements  of  the  ATD's  controls,  by  keeping 
his  hands  and  feet  on  the  controls  the  student  can  be  guided  both  in 
types  and  magnitudes  of  responses  he  will  be  required  to  1  earn. 
Furthermore,  an  ATD  instructor,  freed  from  responsibilities  for 
maintaining  safe  flight  as  he  must  in  an  aircraft,  can  supplement  the 
programmed  guidance  provided  by  the  automated  demonstration  through 
verbal  directions  regarding  the  cues  to  notice,  how  to  interpret  them, 
and  how  to  respond  when  they  appear.  If  the  ATD  design  permits  the 
demonstration  to  be  played  at  slow  speed  and/or  selected  segments  to  be 
repeated,  the  instructor  can  use  these  capabilities  to  focus  on  cue- 
response  coordinations  that  are  particularly  troublesome  to  discriminate 
in  real  time. 
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Many  devices  dc  not  have  instructional  features  that  permit 
demonstrations  to  be  presented  automatically.  In  some  such  devices,  an 
instructor  can  demonstrate  required  performance  manually,  either  by 
operating  flight  controls  located  at  the  instructor  station  or  by 
occupying  a  copilot  position  in  the  device  itself.  Even  in  devices  with 
automated  demonstration,  instructors  may  wish  to  use  these  manual  control 
features  to  demonstrate  something  not  included  in  available  automated 
demonstrations.  (A  similar  capability  exists  for  instructor-conducted 
demonstrations  in  aircraft  with  dual  controls,  of  course,  and  is  widely 
used  during  in-flight  training.) 

Although  not  commonly  used  in  ATDs ,  demonstrations  of  inappropriate 
actions  can  have  a  positive  guiding  value  if  they  are  used  to  highlight 
appropriate  actions.  Some  flight  instructors  recognize  this  point 
intuitively.  For  example,  they  may  say,  "Here's  the  way  some  students 
try  to  do  it;  now  notice  wnere  they  go  wrong."  This  well-founded 
intuitive  approach  to  teaching  discriminations  is  frequently  used  in 
Hight  to  highlight  correct  actions.  It  could  also  be  used  in  some  ATDs 
(e.g.,  those  configured  to  permit  an  instructor  to  occupy  a  copilot's 
position)  that  do  not  have  an  automated  demonstration  capability. 
Inappropriate  performance  could  also  be  demonstrated  automatically  in 
devices  with  that  instructional  feature,  although  no  examples  of  such 
uses  were  found  during  the  current  project.  It  is  important,  of  course, 
that  any  demonstrated  inappropriate  performance  be  clearly  identified  as 
such,  and  that  its  purpose  be  to  aid  the  discrimination  of  correct 
actions. 

Replay  of  a  student's  errors  can  accomplish  a  similar  guidance 
purpose,  but  such  replays  must  await  an  incorrect  performance  that  in 
many  instances  could  and  should  have  been  prevented.  Replay  has  the 
advantage,  however,  of  focusing  on  each  student's  particular  problems  and 
thus  combining  guidance  with  feedback  for  more  effective  training.  While 
a  student  is  guided  during  replay  to  discriminate  correct  cues  and 
responses  he  can  also  observe  the  effects  ( i . e . ,  receive  feedback)  of  his 
earlier  judgments  and  control  responses. 

References  to  coir  niug  guidance  and  feedback  have  sometimes  been 
confusing  to  ATD  instructors,  especially  when  feedback  seems  itself  to 
have  a  guiding  role.  "Guidance"  usually  refers  to  direction  given  a 
student  before  he  attempts  1  learning  task.  "Feedback"  is  Information 
regarding  the  adeq.acy  of  his  performance  after  he  does  the  task. 
Generally,  feedback  can  be  used  to  guide  future  performance,  and  when  it 
Is,  new  attempts  at  correct  performance  should  be  made  while  the  correct 
cues  and  responses  are  still  vivid  in  memory. 

Combined  use  of  guidance  and  feedback  can  be  easily  accomplished  in 
an  ATD  with  freeze  and  "fly  out"  capabilities  associated  with  the 
Record/Playback  Instructional  features.  For  example,  suppose  a  student 
overshoots  a  180°  turn  because  he  starts  the  rollout  too  late.  During 
replay  of  a  recording  of  the  maneuver,  the  instructor  could  Identify  the 
progression  of  cues,  freeze  the  device  at  the  point  rollout  should  have 
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begun,  and  let  the  student  "fly"  it  out.  The  student  thus  gets  feedback 
quickly  on  what  he  did  correctly,  has  the  important  cues  for  initiating 
the  turn  pointed  out,  and  receives  verbal  guidance  from  the  instructor 
for  correcting  his  mistake.  By  "flying  out,"  he  has  an  opportunity  to 
correct  the  mistake  while  the  cues  and  responses  are  still  freshly  in 
mind. 

This  is  not  to  say  that  all  feedback  used  as  guidance  must  be  given 
immediately  after  an  action.  It  is  necessary  only  that  the  information 
received  by  the  student  meaningfully  directs  him  regarding  actions  to 
pursue  or  avoid.  As  explained  in  the  earlier  discussion  of  feedback,  the 
essential  point  is  that  the  cue  interpretation  and/or  response  at  issue 
be  clearly  remembered.  Even  during  debriefing,  feedback  can  have  a 
positive  guiding  effect  if  the  student  can  "relive"  the  experience  in  his 
memory,  i.e.,  recall  the  cues  and  responses  clearly  while  discussing  the 
adequate  and  inadequate  aspects  of  his  performance  with  an  aircrewman 
peer  or  with  an  instructor. 

Guidance  can  be  particularly  useful  in  ATDs  of  relatively  poor 
fidelity.  Whether  cue  and  response  discriminations  learned  in  such  ATDs 
can  be  generalized  to  a  target  aircraft  depends  on  whether  the  student 
has  focused  upon  those  that  are  common  to  both  the  device  and  the 
aircraft,  and  has  learned  to  ignore  aspects  of  ATD  cues  and  responses 
that  do  not  appear  in  the  aircraft.  For  example,  changes  in  apparent 
runway  shape,  varying  look-down  angles  due  to  deviations  in  height, 
increasing  apparent  speed  relative  to  the  ground  while  descending,  etc., 
are  visual  cues  common  to  all  landings.  Such  cues  can  be  represented  in 
many  simulated  visual  scenes,  even  though  the  scenes  themselves  might  be 
of  low  fidelity  and  appear  unrealistic  (e.g.,  symbolic  or  "cartoonish") . 
An  instructor  can  guide  a  student  to  focus  on  the  perspective,  blocking, 
movement,  and  other  cues  that  are  common  to  the  simulated  scene  and  the 
real  world"  scene  and  to  ignore  image  distortions,  color  aberrations,  or 
other  features  of  the  scene  that  are  unique  to  the  simulation. 

The  effectiveness  of  such  instruction  will  depend,  of  course,  upon 
the  ability  of  the  student  to  utilize  similar  cues  in  flight.  With 
sufficient  aircraft  experience,  and  with  guidance  to  notice  similar  cues 
while  flying,  the  generalization  of  appropriate  ATD  visual  cues  can  be 
enhanced,  and  the  danger  of  restricting  cue  recognition  to  device¬ 
peculiar  characteristics  can  be  minimized.  The  student  must  not  be 
permitted  to  use  device-peculiar  characteristics  to  achieve  good 
performance  in  the  device  at  the  expense  of  subsequent  performance  in 
fl ight. 

Guidance  in  Advanced  Training 

As  an  aircrewman's  career  progresses,  he  is  cor  inually  trying  to 
acquire  new,  more  advanced  skills  and  hone  previously  learned  ones.  Each 
time  a  pilot  transitions  to  a  new  aircraft,  e.g.,  from  a  T-37  to  a  T-38, 
and  later  to  a  bomber,  tanker,  cargo  aircraft,  or  fighter,  he  is  in  many 
respects  at  an  "early"  stage  of  learning  new  skills.  He  needs  guidance 


to  help  identify  key  cue  characteristics  and  response  requirements  for 
the  new  aircraft.  In  addition,  past  experiences  in  other  aircraft  will 
have  provided  many  well  mastered  cue  and  response  discriminations  that 
can  be  built  upon  in  acquiring  the  new  skills.  In  such  situations,  the 
experienced  learner  should  be  guided  to  select  previous  discriminations 
that  generalize  readily  to  the  new  tasks.  He  should  also  be  made  aware 
of  past  habits  that  must  be  altered  or  avoided  when  flying  the  new 
ai rc raft. 

For  example,  instructors  often  emphasize  to  students  that  they 
should  focus  on  instruments  when  transitioning  to  a  new  aircraft  such  as 
the  F-4.  The  students  already  know  how  to  interpret  F-4  instrument  cues 
because  they  are  essentially  the  same  as  those  used  in  other  aircraft 
they  have  flown.  By  being  guided  to  use  them,  and,  in  turn,  using  them 
to  help  interpret  new  F-4  cues,  instructors  report  that  the  "feel"  of  the 
new  plane  can  be  learned  by  students  more  quickly. 

The  need  for  guidance  in  aircrew  training  does  not  end  when  skills 
have  been  learned.  Guidance  is  also  needed  to  maintain  previously 
acquired  skills,  or  to  re-establish  levels  of  skills  that  may  have 
deteriorated  during  periods  of  infrequent  practice.  The  most  commonly 
noted  example  of  the  use  of  guidance  not  involving  new  skills  was  the 
scheduling  of  flights  with  instrument  flight  instructors  a  day  or  two 
before  taking  periodic  instrument  flight  checks.  These  flights  provided 
opportunities  for  coaching  (i.e.,  guidance)  where  deficiencies  might  be 
detected  in  order  to  assure  passage  of  the  subsequent  check.  Such 
activities  are  taking  place  in  both  aircraft  and  ATDs. 

Guidance  during  advanced  training  in  modern  ATDs  is  not  necessarily 
dependent  upon  an  instructor.  The  automated  voice  message  guidance 
capabilities  of  several  devices,  e.g.,  automatic  alerts  when  performance 
parameters  are  exceeded  and  instructions  and  commentary  accompanying 
automatic  demonstrations,  were  found  to  be  valued  by  some  experienced 
pilots  who  were  practicing  previously  mastered  skills  when  qualified 
instructors  were  not  present.  In  some  instances,  these  pilots  simply  had 
forgotten  or  were  unable  to  meet  exact  performance  standards.  Or,  they 
had  forgotten  how  to  initiate  a  maneuver,  or  how  to  coordinate  aircraft 
dynamics  with  vectors  required  for  weapons  delivery.  Some  experienced 
aircrewmen  found  that  automated  demonstrations  of  task  performance  aided 
them  by  defining  standards  in  performance  terms,  i.e.,  not  only 
identifying  them  but  showing  how  they  are  achieved.  Automated 
demonstrations  can  also  help  prevent  unproductive  trial -and-error  efforts 
during  reacquisition  of  partially  forgotten  skills,  thus  speeding 
relearning. 

Precautions  in  the  Use  of  Guidance 


Guidance  can  easily  be  overdone.  A  student  can  learn  to  depend  on 
external  guidance  such  as  verbal  directions  from  an  instructor  rather 
than  depending  on  himself.  While  guiding  information  may  be  needed  to 


^ a ; i  1  i tate  skill  acquisition  and  maintenance,  operational  performance 
usi,  not  depend  upon  its  presence. 

/be  most  common  instances  of  overuse  of  guidance  in  ATD  training 
o^'t’fied  during  this  study  were  by  instructors  who  were  overprotecti ve. 
rt'*'  .,  .hey  tried  to  prevent  students  from  making  mistakes  by  giving 

o  me ry  instructions  too  often,  by  dominating  the  controls,  or  by  taking 
( ve  -  concr cl s  ("stick  grabbing")  when  students  deviated  only  slightly 
r_i,:  requirements.  It  is  sometimes  important  that  students  be  permitted 
to  make  mistakes.  Learning  to  discriminate  between  a  correct  action  and 
n.'gnly  similar  but  incorrect  action  can  be  facilitated  if  the  student 
can  experience  Doth,  and  identify  their  differences. 

Guidance  should  be  used  only  when  needed  to  focus  on  cue  and  response 
discriminations  that  the  student  is  not  yet  able  to  make  unaided.  A 
.  ... le  rule  of  thumb  is  available  for  the  use  of  guidance:  Withhold 
gi'ca'.ce  and  note  whether  the  student  progresses  in  performance.  If  so, 
let  him  worn  on  his  own  until  progress  ceases. 

This  rule  is  easy  to  follow  in  an  ATD,  but  often  more  extensive 
guidance  must  be  used  in  aircraft  to  assure  continued  safe  conduct  of  the 
flight.  Some  combat  maneuvers  are  too  dangerous  to  allow  students  to 
attempt  if  they  are  likely  to  make  dangerous  errors:  An  A-7  should  not 
be  put  into  a  departure  condition  by  an  unprepared  student;  an  F-4  should 
not  be  stalled  at  all.  Opportunities  to  recognize  cues  that  lead  to 
tails  thus  must  be  limited  in  the  A-7  and  F-4  aircraft,  and  responses 
-er, aired  for  precision  performance  of  dangerous  maneuvers  must  be 
.arefully  guided.  In  ATCs,  approaches  to  stalls  could  proceed  to 
intolerable  limits,  and  a  student  could  experience  when  he  has  exceeded 
M>afe"  performance  cues.  He  can  learn  how  not  to  execute  a  maneuver  as 
well  as  how  it  should  be  flown.  Safe  performance  in  aircraft  would  then 
be  more  likely,  because  cues  indicating  danger  could  be  more  reliably 
discriminated  by  the  pilot. 


SUMMARY 

This  chapter  discussed  three  factors  of  critical  importance  in 
aircrew  training.  First,  the  role  of  cognitive  training  was  explained, 
including  the  use  of  cognitive  processes  in  developing  knowledge  of 
aircrew  tasks  and  in  aiding  skill  acquisition.  Second,  the  need  for 
aircrew  students  to  obtain  information,  i.e.,  feedback,  regarding  their 
actions,  and  productive  and  nonproductive  uses  of  feedback,  were 
described.  Third,  the  role  and  effective  use  of  guidance  in  ATD  training 
were  addressed,  as  well  as  the  dangers  of  overuse  of  guidance.  These 
discussions  are  summarized  below. 

•  A  meaningful  context  should  be  provided  for  the  learning  of 
skilled  performance  in  the  ATD.  In  recognition  of  the  role  of  mediation 
in  the  acquisition  of  aircrew  skills,  the  student  should  be  given 
information  that  will  enable  him  to  interpret  stimuli,  process  cue 
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information,  and  select  appropriate  responses.  He  should  he  informei 
the  purposes  of  actions,  their  context,  and  types  of  decisions  t;  he-  ••iac_ 
regarding  those  actions. 

•  Mediation  should  be  employed  systematically  in  teaching  n,.-  an:, 

response  discriminations.  The  student  shoul  d  be  guided  as  to  wha.  e  i : 
to  think  about  during  the  performance  of  a  task.  He  shoal c  be  ,'<i' 
conscious  of  what  he  is  experiencing  so  as  to  make  cue  and  •*.  • 

characteristics  stand  out.  For  full  effectiveness,  various  ajJv*  ; 
explanations  and  representations  should  be  coordinated  ■  e»  r  . 

presentations. 

•  Students, _ when _ they  are  practicing^  in  the  ATS,  ,o 

instructed  to  think  about  the  meaning  and  effects  of  thei”  a.  . ; oi;s  wi  v 

respect  to  requirements  for  performance _ in  the  aircraft  ATD  jrastio 

should  be  in  a  context  ot  realism  regardless  of  the  ahgsiu 
characteristics  of  the  device.  Awareness  cf  such  provides  a  b  s:r  foi 
transferring  discriminations  of  cues  and  responses  t,.  cii.raft 
performance. 

•  Briefing  for  ATD  sessions  shoul d_ anticipate  and  guard  ags-ipti  a 

student's  relating  what  he  does  during  ATD  practice  on1y_  to~thi  ivY. 
i  tsel  f .  Over  and  above  expl  anatiotis  of  performance  requiremc  fv. 

student  should  be  prepared  through  briefings  tc  imagine  the  occurs .  o‘ 

cues  missing  in  the  ATD,  to  focus  on  tnost  aspects  of  cues  presen;  :n  the 
ATD  that  also  appear  in  actual  flight,  and  to  ignore  aspects  net 
available  in  flight. 

•  Recognizing  that  transfer  is  a  meciaticnal  process,  a  major  goa1 
in  ATD  training  should  be  to  exploit  the  types  and  uses  of  mediation  that 
maximize  transfer.  Since  ATD-ai reraf'e  cues  and  responses  are  not 
identical,  it  is  important  that  cues  common  to  both  ATDs  and  aircraft  be 
discriminated  from  those  that  are  unique  to  the  ATD.  Similarly,  the 
usable  components  of  ATD  responses  associated  with  the  common  cue 
meanings  must  be  discriminated  from  those  that  are  specific  to  .he  ATD. 
Both  of  these  types  of  discriminations  result  in  transfer,  and  cognitive 
mediation  is  an  essential  means  of  learning  to  make  them. 

•  Feedback  determines  the  natur e  and  extent  of  di scri mi  nations  that 
will  be  learned.  Uses  of  feedback  should  be  designed  to  ensure 
discriminations  of  task-intrinsic  rues,  their  interpretation,  and  the 
appropriate  actions  necessary  for  skill  performance. 

•  Feedback shoul_d  focus  on  specific  aspects of  cognitive, 

perceptual,  and  motor  actions  that  must  be  discriminated.  Generally  the 
learning  of  subtle  characteristics  of  actions  depends  on  the  occurrence 
of  feedback  directed  toward  the  specific  characteristics  that  are  to  be 
di scrimi rated. 

•  Augmented  and  supplemental  feedback  should  be  used  only  when 
task-i ntri nsi_c_ feedback  cannot  be  di scriminated,  and  its  use  should  be 
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specifically  for  the  purpose  of  teaching  cue  and  response  discri mina ti ons 
intrinsic  to  the  aircraft  task.  The  need  for  such  feedback_fs  determined 
by  the  student's  ability  to  discriminate  and  use  intrinsic  feedback.  It 
should  not  be  continued  to  the  point  that  the  student  becomes  dependent 
on  feedback  that  will  not  occur  in  aircraft. 

•  Timely  supplemental  or  augmented  feedback  should  be  used  to 
signal  the  availability  of'intrinsic  feedback.  By  associating  stimulus 
conditions  with  information  provided  through  supplemental  and  augmented 
feedback,  the  cue  information  in  the  stimuli  can  be  learned. 

•  The  timing  of  augmented  and  supplemental  feedback  should  be 
determined  by  requirements  for  learning  discriminations  of  task-intrinsic 
cues  and  actions"!  How  soon  such  feedback  should  be  provided  after  a  cue 
interpretation,  judgment,  or  motor  response  depends  upon  how  long  all 
pertinent  characteri sties  of  the  action  can  be  vividly  remembered. 

•  When  intrinsic  feedback  is  needed  as  a  signal  for  a  subsequent 
action,  ATOs  must  provide  it  quickly  enough  to  avoid  disrupting  the 
action.  Coordinations  of  control  movements  with  many  motion,  visual,  and 
Instrument  cues  require  immediate  feedback  if  the  movements  are  to  be 
smooth  and  their  component  responses  timed  accurately. 

•  Feedback  will  help  maintain  student  motivation  if  it  ensures 
progress.  Learner  satisfaction  depends  on  progress.  When  the  student  is 
unable  to  perceive  that  progress  has  been  made,  supplemental  feedback, 
such  as  favorable  comments  by  the  instructor,  should  be  used. 

•  The  purpose  of  guidance  is  to  focus  the  student's  attention  on 
correct  cue  and  response  discrimination  and  to  avoid  incorrect  cue 
interpretations  and  actions.  Guidance  speeds  up  learning  when  it 
prevents  irrelevant  efforts  by  the  student  and  focuses  his  attention  on 
pertinent  cues,  their  meanings,  and  indicated  actions. 

•  To  avoid  a  dependence  on  guidance,  it  should  not  be  used  when  the 
student  is  able  to  make  the  required  di scrimination  without  help.  The 
goal  is  to  teach  self-sufficiency  in  task  performance.  Therefore ,  the 
student  should  practice  as  much  as  possible  selecting  his  own  cues, 
determining  their  meanings,  and  selecting  appropriate  actions.  A  rule  of 
thumb  for  employing  guidance  is  to  use  it  only  when  the  student  cannot 
discriminate  appropriate  cues  and  responses  as  indicated  by  an 
unsatisfactory  rate  of  progress  when  left  on  his  own. 

•  Contrasting,  through  guidance,  desirable  with  undesirable  cue 
interpretations  and  responses  can  highlight  critical  cues  and  responses 
when  the  discriminations  to  be  learned  are  difficult.  The  emphasi s 
shoul d  be  on  desi rable  aspects  of  actions,  wi th  undesirable  aspects 
clearly  identified  as  such  and  used  only  when  contrast  is  needed. 

•  When  feedback  is  used  primarily  in  a  guiding  role,  it  should 
occur  as  soon  as  practical  after  the  action,  and  the  student  should 
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repeat  the  action  without  undue  delay.  To  guide  an  action  in  retrospect, 
the  cues  and  components  of  the  action  must  be  vivid  in  memory  when  the 
feedback  is  given,  and  correct  performance  should  be  attempted  while  it 
is  freshly  in  mind.  A  novice  may  need  immediate  feedback  followed  by 
additional  efforts;  but  for  an  advanced  student,  delayed  feedback  even 
during  a  debriefing  can  have  a  guiding  function  if  its  relation  to  cues 
and  responses  is  clear  and  remains  so  for  subsequent  trials. 

•  Guidance  should  be  used  during  advanced  training  when  needed  tc 
focus  students'  attention  on  new  cue  and  response  discri mi  nations,  or  on 
previously  mastered  discriminations  that  can  be  generalized  to  new  tasks. 
Transition  training  especially  involves  new  variations  of  familiar  cues 
and  responses.  Similarities  to  previous  cues  and  responses  as  well  as 
differences  should  be  pointed  out. 

•  Guidance  can  be  valuable  for  experienced  aircrews  when  the  skills 
have  deteriorated  appreciably,  or  when  it  can  define  or  clarify  standards 
for  performance.  Once  a  skill  has  been  mastered,  an  aircrewman  will 
usually  have  the  discriminative  capacity  to  profit  from  ATD  practice 
without  external  guidance.  However,  if  cue  and  response  discriminations 
have  deteriorated  seriously,  some  guidance  may  be  beneficial.  Also,  some 
experienced  and  fairly  proficient  aircrews  have  found  programmed 
demonstrations  valuable  because  they  illustrate  the  standards  for  ideal 
performances. 

•  Guidance  should  focus  on  aspects  of  ATD  skill  performance  that 
are  transferable  to  aircraft,  or  can  promote  transfer.  These  aspects  may 
be  cues  and  responses  per  se,  or  generalizable  aspects  of  them.  In 
process,  students  should  be  guided  to  notice  ATD-specific  aspects  of  cues 
and  responses  so  they  can  be  discriminated  and  negative  transfer  avoided. 
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CHAPTER  V 

CONSIDERATIONS  IN  STRUCTURING  ATD  TRAINING 


The  two  preceding  chapters  described  the  learning  process  as  it 
relates  to  ATD  utilization  (Chapter  III)  and  the  conduct  of  ATD  training 
(Chapter  IV).  The  present  chapter  addresses  factors  to  be  considered  in 
the  structuring  of  ATD  training.  These  factors  are:  (1)  establishing 
priorities  for  ATD  uses;  (2)  sequencing  ATD  training  in  relation  to 
academic  and  flight  training;  (3)  allocating  training  among  various  types 
of  available  ATDs;  (4)  organizing  tasks  into  practice  sessions;  and  (5) 
determining  the  frequency  and  duration  of  individual  ATD  training 
sessions. 


PRIORITIES  FOR  ATD  SCHEDULING 

In  addition  to  introducing  aircrews  to  operational  performance 
requirements  at  both  UPT  and  combat  crew  training  levels,  ATDs  have  been 
used  for  continuation  and  refresher  training,  for  upgrading  and  teaching 
specia’  skills,  and  for  remedial  training.  At  some  locations  they  are 
being  used  in  training  aircraft  maintenance  personnel,  loadmasters, 
traffic  controllers,  ATD  instructors,  command  personnel,  and  even  flight 
surgeons. 

Scheduling  ATDs  for  training  thus  needs  to  be  responsive  to  two 
types  of  priorities.  One  type  involves  different  kinds  of  training, 
(e.g.,  combat  crew  training  vs  continuation  training  vs  instructor 
training  vs  maintenance  personnel  training).  The  second  involves 
different  training  needs  within  a  training  group  (e.g.,  aircraft  control 
vs  emergency  procedures  vs  low  level  navigation). 

Both  types  of  priorities  are  concerned  with  assuring  that  the  ATD  is 
utilized  appropriately  during  routine  use  as  well  as  when  its 
availability  changes.  Such  changes  may  occur  when  scheduled  or 
unscheduled  device  maintenance  must  be  performed  during  time  periods 
normally  used  for  training,  or  when  fluctuations  in  the  flow  of  students 
through  the  training  pipeline  make  increased  or  decreased  demands  on  ATDs 
and  other  training  resources.  These  and  other  such  events  can 
necessitate  the  reallocation  of  training  activities  normally  scheduled 
for  ATDs  to  other  resources,  or  can  make  the  devices  available  for 
training  normally  conducted  by  other  means.  If  periods  of  increased 
device  availability  can  be  anticipated,  training  groups  that  could 
benefit  from  ATD  training  can  be  identified  and  training  scheduled  for 
them  to  take  up  the  slack.  Likewise,  in  times  of  high  demand  by  top 
priority  groups,  alternative,  less  efficient  resources  could  be  used  by 
those  lower  in  priority. 

It  is  desirable  to  establish  priorities  for  training  tasks  on  the 
basis  of  the  kinds  of  training  that  can  be  done  best  in  each  available 
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device.  Alternative  resources,  including  aircraft,  should  be  identified 
for  each  task  to  be  trained,  and  the  relative  effectiveness  and 
efficiency  of  each  resource  estimated.  Trade-offs  could  then  be  made 
between  priorities  and  alternative  ATDs,  with  cost  helping  to  determine 
the  trade-offs.  ATDs  could  be  scheduled  so  as  to  be  available  for 
various  training  activities  and  groups  according  to  an  optimum  priority 
grouping  of  tasks,  with  media  alternatives  and  costs  taken  into 
consideration.  Such  systematic  planning  for  the  use  of  ATD  resources  was 
not  found  to  have  taken  place  at  any  of  the  units  surveyed  during  the 
present  project. 

It  is  important  to  establish  priorities  for  use  of  ATDs  before  the 
need  for  use  of  alternative  resources  arises.  Course  syllabi  should 
reflect  such  priorities  by  identifying  alternative  training  resources  to 
be  considered  for  use.  With  adequate  forewarning,  instructors  could  be 
trained  to  deal  with  the  contingencies  governing  uses  of  ATDs  as  their 
availability  varies  and  be  prepared  to  use  these  devices  and  other 
resources  as  necessary. 

Fixed  procedures  are  not  available  for  making  these  trade-offs  and 
developing  alternative  schedules.  The  general  guidance  contained  in  the 
section  of  this  chapter  dealing  with  the  allocation  of  training  to  ATDs 
will  be  useful  in  making  the  trade-offs,  as  will  the  discussion  in  the 
section  dealing  with  the  separation  of  tasks  for  practice.  In  addition 
to  this  guidance,  the  trade-offs  must  be  based  upon  the  experience  of 
training  personnel  with  the  devices  and  training  requirements  involved. 


SEQUENCING  ATD  AND  OTHER  TRAINING 

The  contributions  of  ATD  training  varies  depending  upon  how  it  is 
integrated  with  aircraft  and  academic  training.  Four  considerations  are 
of  concern:  (1)  Training  in  ATDs  can  provide  concrete  meaning  for 
theoretical  knowledge  and  timely  practice  in  applying  that  knowledge;  (2) 
academic  instruction  can  sometimes  be  more  meaningful  if  it  follows  ATD 
practice;  (3)  transfer  of  ATD  training  to  aircraft  often  requires 
practice  in  aircraft  soon  after  ATD  learning;  and,  (4)  some  ATD 
instruction  will  be  more  meaningful  if  it  is  preceded  by  exposure  to  the 
tasks  in  aircraft. 

Application  of  Theoretical  Knowledge 


Since  the  purpose  of  most  academic  training  is  to  convey  information 
(knowledge)  necessary  to  the  development  of  skills  in  ATDs  or  aircraft, 
it  is  appropriate  that  it  usually  be  scheduled  to  occur  before  those 
skills  are  first  practiced  in  the  device  or  aircraft.  That  is,  ATDs 
normally  should  provide  practice  in  use  of  knowledge  and  theoretical 
concepts  presented  during  academic  training  after  the  concepts  have  been 
learned  at  a  verbal  level.  The  ATD  scheduling  question  is  how  long  after 
academic  training  will  such  practice  be  most  fruitful? 
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The  answer  must  be  in  terms  of  the  retention  capabilities  of 
students.  Factors  to  consider  in  retention  are  the  vagueness  of 
distinctions  among  tasks  when  they  are  understood  only  verbally;  the 
length  of  time  the  student  can  remember  whatever  discriminations  he  has 
learned;  and,  since  tasks  currently  being  practiced  but  not  yet  mastered 
can  interfere  with  new  learning,  the  amount  of  new  information  that  can 
be  presented  during  academic  instruction  prior  to  ATD  practice  without 
loss  of  efficiency. 

There  are  no  simple  rules  to  follow  in  considering  these  factors. 
Decisions  in  a  particular  training  program  must  be  based  upon  the 
judgment  of  personnel  familiar  with  the  training  being  conducted  and  the 
experience  levels  of  the  students  involved.  The  counsel  of  experienced 
instructors  and  educational  specialists  obviously  can  be  helpful  in 
making  such  judgments. 

A  point  to  remember  when  making  decisions  Is  that  opportunities  to 
apply  knowledge  acquired  during  academic  training  should  not  be  delayed 
for  very  long  for  novices.  They  will  not  have  sufficient  prior  mastery 
of  related  knowledge  to  prevent  concepts  from  getting  mixed  up.  As  the 
related  experiences  of  the  student  increases,  however,  greater  delays  can 
be  tolerated  without  ill  effects.  Thus,  the  experience  level  of  the 
student  is  a  factor  in  scheduling  ATD  practise,  with  the  less  experienced 
student  needing  practice  sooner  after  relevant  knowledge  has  been 
presented  to  him. 

ATD  Training  First 

The  second  consideration  concerns  the  need  for  ATD  experience  prior 
to  academic  training.  There  are  situations  in  which  prior  experience  or 
familiarity  with  an  application  of  theoretical  concepts  will  make  it 
easier  to  understand  the  knowledge  to  be  learned.  Dive  angles  required 
for  specific  weapons  delivery  tasks,  for  example,  cannot  be  meaningful 
verbally  until  after  the  student  has  had  some  experience  with  the  visual 
images  and  kinesthetic  feedback  that  accompanies  performance  of  these 
tasks.  The  point  is  that  concrete  anchors  for  knowledge  should  be 

provided  when  they  are  needed;  and  for  some  academic  instruction  to  have 
meaning,  it  is  necessary  that  the  student  at  least  have  limited 
experience  with  the  actual  cues  and  intrinsic  feedback  associated  with 
the  tasks  concerned. 

When  ATDs  can  provide  the  experiences,  specific  ATD  practice  should 
be  scheduled  prior  to  or  concurrent  with  academic  exposure  to  the 
knowledge  of  concern.  For  example,  ground  effects  can  be  explained,  but 
the  student  will  probably  still  be  somewhat  startled  the  first  time  he 
experiences  them.  Experience  with  ground  effects,  their  onset  and 
disappearance,  will  enable  the  student  in  his  imagination  to  integrate 
control  adjustments  with  conditions  under  which  they  occur. 
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Integrating  ATD  and  Aircraft  Practice 


The  third  consideration  that  has  implications  for  scheduling 
concerns  the  interspersing  of  ATO  and  aircraft  practice.  Training 
conducted  in  ATOs  must  involve  some  symbolic  cues  and  responses,  because 
the  fidelity  of  all  current  state-of-the-art  devices  is  less  than 
perfect.  For  lower  fidelity  devices,  a  large  proportion  of  the  cues  and 
actions  required  for  performance  of  operational  tasks  are  often  symbolic. 
(Reliance  upon  symbolic  cues  and  responses  when  using  ATDs  is  discussed 
extensively  in  Chapters  III  and  IV.) 

Just  as  ATDs  can  provide  concrete  anchors  for  understanding  academic 
or  theoretical  concepts,  timely  aircraft  practice  can  provide  the  actual 
cues  and  responses  needed  to  anchor  ATD-based  symbolic  knowledge  in 
operational  reality.  Considerations  involved  in  the  timing  of  such 

practice  in  aircraft  are  similar  to  thore  described  above  for  the 
practice  in  ATDs  of  knowledge  and  theoretical  concepts  learned  during 
academic  instruction.  That  is,  the  delay  should  not  exceed  the  retention 
capabilities  of  the  students.  The  more  symbolic  the  training  conducted 
in  the  ATD,  the  shorter  the  delay  should  be.  Likewise,  the  less 

experienced  the  students,  the  shorter  the  delay  should  be.  Thus,  rela¬ 

tively  inexperienced  students  undergoing  training  in  relatively  low 
fidelity  devices  should  be  provided  opportunties  to  employ  the  ATD- 

developed  skills  in  the  aircraft  as  soon  as  practical.  As  student 
experience  levels  increase  and  higher  fidelity  part- task  trainers  and 
simulators  are  employed  for  skill  acquisition,  longer  delays  can  be 
tolerated  without  unacceptable  decrements  in  performance.  In  any  event, 
delays  should  be  no  longer  than  necessary  to  accommodate  the  practical 
aspects  of  scheduling  all  training  activities. 

Aircraft  Training  First 

The  fourth  consideration  concerns  the  fact  that  for  some  tasks, 
aircraft  experiences  may  be  needed  before  ATD  practice  so  that  the  latter 
can  have  the  meaning  necessary  for  effective  training.  Part  of  the 
discussion  in  a  later  section  of  this  chapter  on  allocating  training 
among  ATDs  is  especially  pertinent  here.  It  is  explained  there  why  an 
experienced  aircrewman  can  derive  more  training  benefits  from  a  low 
fidelity  device  than  a  novice  can.  Basically,  the  reason  is  that  the 
symbolic  actions  required  to  perform  some  tasks  in  some  ATDs  already  have 
concrete  meanings  for  the  experienced  aircrewman  because  he  "has  been 
there  before."  Therefore,  the  more  novel  the  task  to  be  learned  in  the 
ATD--the  purpose,  the  cues,  the  actions— and  the  lower  the  ATD  fidelity, 
the  greater  the  need  for  aircraft  experience  prior  to  ATD  practice. 
Perceived  altitude  at  roundout,  for  example,  is  vague  at  best  to  one  who 
is  not  familiar  with  ground  cues  for  height;  "feel"  cues  characteristic 
of  a  float  are  foreign  to  one  who  has  never  flown  an  aircraft  in  this 
situation;  control  corrections  to  counteract  torque  or  adverse  yaw  are 
Just  textbook  instructions  until  the  feel  of  maintaining  corrections 
becomes  associated  with  actual  perceptions  of  their  effect  on  the 
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behavior  of  an  aircraft.  Schedules  should  permit  tasks  involving 
unfamiliar  cues  and  reponses  to  be  attempted  or  demonstrated  in  aircraft 
before  the  tasks  are  practiced  in  ATDs. 

The  importance  was  noted  in  Chapter  IV  of  pointing  out  to  students 
any  discrepancies  between  ATDs  and  aircraft  during  training  in  the 
devices.  However,  these  differences  may  not  be  understood  when  an 
instructor  describes  them.  Again,  this  is  more  likely  to  happen  with 
inexperienced  students  than  with  more  experienced  ones.  To  discriminate 
differences  in  his  imagination,  he  must  be  able  to  experience  *he 
differences  in  his  imagination.  If  past  experience  is  not  sufficient  to 
define  the  differences  for  him,  and  if  the  experience  can  be  obtained 
only  in  an  aircraft,  he  will  need  some  aircraft  experience  either  before 
or  interspersed  with  ATD  training.  The  "dollar  ride"  given  at  the 
beginning  of  some  aircrew  training  programs,  if  carefully  structured, 
can  provide  such  experience. 


ALLOCATION  OF  TRAINING  AMONG  ATDS 

ATDs  can  vary  in  effectiveness  and/or  efficiency  for  teaching  a 
particular  skill.  In  such  cases,  a  plan  for  the  use  of  specific  ATDs 
should  reflect  the  capabilities  of  individual  devices.  In  addition, 
occasions  arise  when  two  or  more  different  types  of  ATDs  are  available 
and  are  essentially  equivalent  in  value  for  teaching  a  particular  skill. 
If  so,  factors  other  than  device  capabilities  can  be  considered  in  their 
use.  The  purpose  of  this  section  is  to  examine  these  two  aspects  of  ATD 
utilization. 

Utilization  Based  on  ATD  Capabi 1 i ties 


One  consideration  in  ATD  utilization  is  the  extent  to  which  verbal 
instruction  or  otner  symbolic  cues  and  responses  can  substitute  for  or 

supplement  cue  and  response  fidelity  in  a  particular  device.  A  second 

consideration  is  the  prior  experience  of  the  student.  A  third  is  the 
amount  of  practice  required  to  master  the  skill  or  skill  segment  of 
interest.  A  fourth  is  the  general i zabi 1 i ty  of  the  skills  being 

practiced. 

Substitution  of  Cues  and  Responses 

This  consideration  is  primarily  concerned  with  needs  for  physical 
fidelity  in  ATDs.  Strict  fidelity  is  not  required  if  appropriate  cues 
and  responses  can  be  visualized  by  the  student.  For  example,  low  physical 
fidelity  devices  can  be  used  to  teach  many  procedural  tasks,  because 
students  can  vividly  imagine  the  events  that  make  up  the  procedures, 
e.g.,  using  a  radio,  sighting  a  target,  positioning  a  control,  observing 
the  status  of  a  light,  and  determining  the  value  indicated  on  a  meter  or 
display.  Similarly,  students  can  comprehend  the  actions  that  would 

affect  these  events  without  actually  having  to  engage  in  them,  and  they 
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can  imagine  the  consequences  of  those  actions  (i.e.,  feedback)  without 
its  actually  occurring. 

As  one  moves  from  procedural  tasks  to  aircraft  control  type  tasks, 
however,  students  will  not  as  readily  be  able  to  imagine  the  feel, 
pressures,  and  other  types  of  intrinsic  feedback  that  their  actions 
normally  provide,  so  the  device  generally  must  be  of  higher  fidelity. 
That  is,  the  students  must  be  able  to  engage  physically  in  the  action 
involved  in  performing  a  task,  and  the  feedback  critical  to  accurate 
performance  of  those  tasks  must  be  provided. 

P rior  Exper i e nc e 

In  allocating  training  to  ATDs  that  vary  in  fidelity,  the  experience 
of  the  student  must  be  taken  into  consideration.  As  a  general  rule,  the 
more  experienced  the  student  in  interpreting  cues  and  responding  to  them, 
the  more  he  can  employ  imagery  during  his  training  and  the  less  dependent 
he  is  upon  device  fidelity  in  learning  a  particular  skill.  An 
experienced  pilot  can  derive  more  training  value  from  a  low  fidelity 
device  than  can  a  novice  because  he  can,  in  his  imagination,  more  readily 
"see"  an  instrument  change,  "hear"  an  engine  start,  "feel"  an 
acceleration,  etc. 

It  does  not  follow,  however,  that  high  fidelity  is  not  desirable  or 
needed  when  the  students  are  experienced  aircrewmen.  High  fidelity 
simulation  is  viewed  by  most  pilots  as  increasingly  important  as  their 
proficiency  and  experience  levels  increase  because  the  skills  to  be 
practiced  require  coordination  of  cue- response  systems  where  realistic 
kinesthetic  and  visual  feedback  must  be  precisely  integrated.  Such  is 
the  case  in  continuation  training  and  maintenance  of  operational 
proficiency,  and  the  device  selected  for  these  purposes  must  provide 
realistic  control  dynamics  and  task-intrinsic  feedback  regardless  of 
prior  learner  experiences. 

Practice  Reg ui rements 

In  allocating  training  to  various  part-task  ATDs,  portions  of  tasks 
that  are  commonly  difficult  to  learn  should  receive  specific  attention, 
and  ATDs  should  be  sought  in  which  additional  practice  on  those  tasks  can 
be  provided  without  requiring  that  time  be  spent  on  skill  components 
already  mastered.  For  example,  vectoring  for  an  aerial  intercept  is 
relatively  difficult  for  a  student  when  the  task  is  first  introduced. 
However,  he  will  not  require  much  practice  actually  piloting  the  aircraft 
at  such  a  point  in  his  training,  since  he  presumably  already  will  be 
relatively  proficient  at  that  task.  Extensive  training  on  the  intercept 
task  itself  can  be  conducted  in  a  part-task  trainer  that  permits  practice 
of  vectoring  (e.g.,  an  operational  flight  trainer),  and  when  performance 
standards  are  met,  the  intercept  skills  can  be  integrated  with  previously 
existing  aircraft  control  skills  in  a  weapons  systems  trainer  or  other 
device  that  permits  the  complete  task  to  be  practiced. 
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An  analysis  of  the  learning  tasks  using  the  suggestions  for  task 
separation  discussed  in  a  later  section  of  this  chapter  could  identtfy  a 
number  of  aircrew  skills  that  could  be  practiced  separately  from  other 
tasks.  These  different  task  elements  and  segments  could  then  be 
allocated  to  different  part-task  ATDs  in  such  a  way  that  the  overall 
capabilities  of  each  type  device  could  be  used  efficiently.  Mission 
simulators  and  similar  complex,  "sophisticated"  devices  would  then  be 
used  primarily  for  sophisticated  purposes,  i.e.,  to  provide  practice  in 
integrating  the  various  task  parts  into  meaningful  whole-task 
performance. 

General izability  of  Skills 

A  major  concern  in  ATD  training  is  the  general izability  of  the 
skills  being  learned.  Skill  general ization  depends  upon  having  learned 
to  discriminate  key  cues  and  responses.  Provisions  for  cues  and 
responses  that  are  keys  to  performance  of  particular  tasks  should  be  a 
major  basis  for  the  utilization  of  ATDs.  When  one  focuses  upon  the  cues 
and  responses  that  underlie  performance  of  the  tasks  to  be  practiced,  he 
then  can  select  an  ATD  with  those  cue-response  characteristics  and 
allocate  practice  of  such  tasks  to  it. 

The  significance  of  the  general izability  of  skills  training  for  the 
allocation  of  training  to  particular  ATDs  is  illustrated  in  the  use  of  a 
T-40  trainer  for  practice  of  instrument  flight  skills  by  B-52  and  C-135 
pilots.  It  did  not  matter  so  far  as  performance  in  the  aircraft  was 
concerned  that  the  T-40  was  radically  different  in  cockpit  configuration 
and  dynamics  from  these  pilots'  operational  aircraft.  The  cues  and 
responses  that  are  keys  to  the  instrument  flight  task  were  present  in 
these  devices.  Therefore,  instrument  flight  training  could  be  conducted 
for  the  pilots  in  the  T-40  instrument  flight  trainer  while  other  training 
was  allocated  to  ATDs  that  simulated  their  operational  aircraft. 

This  example  also  illustrates  how  aircrew  skills  that  are  not 
dependent  upon  a  particular  aircraft  can  be  mastered  independently  of 
aircraft  configuration,  and,  hence,  in  ATDs  that  do  not  necessarily 
correspond  to  a  particular  operational  aircraft.  Skills  are  learned  in 
vehicles  (ATDs  and  aircraft),  but  properly  mastered,  they  become 
relatively  independent  of  given  vehicles.  They  become  part  of  the  skills 
inventory  of  the  aircrewman  who  has  learned  them,  and  can  then  be  used  in 
a  variety  of  vehicles. 

ATDs  not  targeting  specific  aircraft  were  looked  upon  with  disdain 
by  many  of  the  training  personnel  interviewed  during  the  present  project, 
as  were  low  fidelity  devices  in  general.  They  should  not  be.  Clearly, 
the  more  tasks  that  can  be  performed  in  a  device,  the  greater  its 
potential  training  value.  However,  even  devices  in  which  only  a  few 
tasks  can  be  performed  can  have  training  value  for  these  tasks  equal  to 
the  value  of  more  complex  devices.  If  general izable  cues  and  responses 
are  Involved,  a  skill  learned  in  a  low  fidelity  part-task  trainer  will  be 
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just  as  valuable  m  tfu-  aircraft  as  ".ill  ti.e  same  skill  learned  to  the 
same  standards  in  a  high  fidelity,  full  mission  weapons  system  trainer. 

Utilization  of  tqually  Useful  Aids 

There  are  situations  in  which  AIDS  of  different  levels  of  fidelity 
and  complexity  -.rc  available  for  use.  and  tney  are  essentially  equal  with 
respect  to  th>  v lininc  to  '  •  ..'■>•  ducted.  in  fact,  such  occasions  arise 
frequently .  for  ,•<  ample,  ’  •  >t  combat  crew  training  units,  both  c;''s 

air  >'T$  ar*-  ava  .  *  <_•  ana  m  "*enti  al  ly  equal  ly  suitable  with  respect 

to  the  tra;  if  :  o'  it  p'  s: 'dares  tasks.  In  deciding  which  device  *r 

use,  tne  v.ri ter'  *.  W  :•..<!  *  ove  for  the  utilization  of  avc  labie  A’i;s 
do  net  apply,  fnere  ... r ..  i  iher  factors  to  consider,  however. 

One  such  factor,  of  course,  is  cost.  Insofar  as  operational  costs 
vary  among  devices,  and  device  "wear  and'  tear"  is  increased  by  use. 
capital  and  maintenance  costs  become  important  considerations .  Cost  we; 
observed  to  be  a  major  consideration  in  allocating  training  objective-:, 
among  ATD$  at  most  of  the  units  surveyed  during  the  present  project. 
Cost  is  also  specified  as  a  major  consideration  in  making  seen 
allocations  in  ISO  and  other  training  program  development  manual:.,  e . 5 . , 
Af  P  5o-b . 

It  was  the  opinion  of  the  project  staff,  however,  chat  ccst  was 

sometimes  weighted  too  heavily  in  decisions  concerning  the  use  of  ATI's . 
Instances  were  note d  in  wrnch  pilots’  preferences  among  available  ATDs 
were  ignored  in  order  to  use  a  relatively  lew  cost  device,  even  though  a 
preferred  but  more  costly  device  was  available.  The  negative  effect  01 
attitudes  toward  AID  training  resulting  fro  -  this  practice  , .,ay  well  have 
outweighed  any  cost  advantage  realized. 

While  cost  must  be  considered  in  any  resource  allocation  decision, 
it  should  not  be  tne  only  factor  considered  in  utilizing  ATDs. 
Consideration  should  also  be  given  to  the  preference  of  tne  aircrewmeu 
who  will  oe  trained  and  of  the  instructors  who  w’li  administer  the 

training.  The  ATD  preferred  by  these  personnel,  in  most  instances,  is 
tne  hi^ncst  fidelity  and  most  nearly  full-task  device  available.  Low 

fidelity  procedures  trainers  are  usually  least  preferred,  regardless  of 
their  demonstrat.-d  training  value  or  lower  eperatira  cost. 

It  is  iinportwit  that  user  pr«.  forer-ce  be  given  consideration. 

Par i ;cul arly .  situations  should  be  avoided  in  which  a  less  preferred 
device  is  requi rec  to  be  used,  but  ar.  equally  suitable  preferred  device 
remains  unused.  The  influence  of  such  ATD  utilization  on  instructor  and 
studerf  attitudes  toward  training  management  was  observed  to  be  negative 
in  several  units  where  this  situation  was  observed.  (’Tore  will  be  said 
about  AID  user  attitudes  in  Chapter  VII.) 


A  i  do  ration  in  the  design  and  conduct  of  any  ATD  training 

activity  s  tne  need  to  organize  the  tasks  to  be  learned  into  practice 
session  .  Whether  motor  skills,  procedural  skills,  communications 
skills,  etc.,  or  more  typically,  a  combination  of  skill  types,  are  to  be 
learned,  t.j  points  stand  out:  (1)  cue  and  response  complexity  should 
be  reduced  in  early  stages  cl  the  development  of  a  skill;  and  (2)  skills 
should  eventually  be  practiced  in  situations  representing  the  full 
complexity  of  ' porat i onal  performance.  Two  questions  follow  from  these 
points:  How  should  tasks  be  separated  for  practice  so  that  their 

complexity  car.  be  reduced  while  learning  the  component  tasks?  How  can 
training  ensure  the  eventual  integration  of  tasks  learned  separately? 
The  subsections  which  follow  address  these  questions. 

$ epar a_t i_n q  rusks  f o r_P rat  lice 

Ever,  the  lost  routine  casks  required  in  aircrew  performance  involve 
using  information  from  many  cues  and  making  many  responses.  Landing,  for 
example,  consists  of  maintaining  precise  .o.otrol  of  an  aircraft  during  a 
series  of  sequentially  dependent  action-:  while  simultaneously  performing 
actions  identified  on  a  landing  checklist,  scanning  the  area  for  other 
aircraft,  communicating  via  radio  wit,  controlling  agencies,  modifying 
flight  according  to  information  received  the  cos-,  uni  cations,  etc.  An 
experienced  pilot  performs  these  tasks  u.rt  rr  less  mechanically.  He 
d'vs  'i.;  have  to  think  about  each  a?  ana  response  in  order  to  perform  it 
correctly.  p.  r  to  a  novice,  landing  consists  of  many  tasks  involving 
ties  and  -e spouses  that  are  not  yet  under  control  well  enough  to  be 
performed  individually  without  error,  let  alone  as  part  of  a  single 
larger  tasx, 

Oftti  the  '«st  efficient  way  to  learn  complex  tasks  is  to  practice 
meaningful  ;..jrts  or  subtasks,  in  relative  isolation  from  each  other. 

Thus,  p:-*s  o*  :  complex  task  should  be  separated  from  others  if  they  can 

he  lo-vned  '  o*  _  easily  when  pract:  e.l  separately.  For  example,  radio 

cnmrm,r  i -.a '  i on  to'  t.-e  Teamed  mor.;  easily  when  separated  from  aircraft 

conf'c'  J. ring  , ni tial  skill  acquisition,  and  discriminations  o'  features 
of  a  wirgman's  aircraft  that  will  serve  as  positional  cues  during 
formation  flic,  it  can  be  learned  more  easily  when  practiced  separately 
from  the  ta:A  r  scanning  for  ether  aircraft.  Likewise,  a  task  should  be 
separated  fr.-m  others  when  such  separation  will  facilitate  cue 
discernment.  Fo*"  example,  aural  cues  indicating  engine  mal performance 
might  not  be  aisceruiji-  when  p-acticed  together  with  other  tasks  having 
unusual  sound  cues. 

Avoidance  of  inter-task  interference  during  learning  is  not  the  only 
reason  fc -  sfv<*ration  cc  tasks,  although  it  is  probably  the  most 
important  one.  Other  rasens  fer  separating  tasks  for  practice  include 
the  need  to  limit  the  h:  noth  of  practice  sessions,  the  convenience  of 
dealing  with  -- it, subject  matter  at  one  time,  aid  the  availability  of 


ATDs  in  which  only  selected  tasks  can  be  trained  (so-called  "part-task" 
trainers) . 

There  are  rules  of  thumb  that  can  be  used  for  identifying  tasks  for 
separate  practice.  (1)  Tasks  requiring  small  amounts  of  practice  should 
be  separated  from  tasks  requiring  larger  amounts.  For  example,  landing 
requires  more  practice  than  taking  off,  so  landing  in  an  ATD  can  be 
isolated  from  the  normal  takeoff-landing  sequence  for  concentrated 
practice.  (2)  Tasks  that  can  be  mastered  more  easily  when  practiced 
repeatedly  should  be  separated  from  others.  For  example,  aircraft 
control  during  formation  flight  can  be  separated  from  other  mission 
activities;  operation  of  EW  equipment  can  be  practiced  separately  from 
aircraft  control .  (3)  Tasks  with  "natural"  break  points  can  be 

conveniently  divided  into  separate  segments  for  practice.  For  example, 
probe  insertion  provides  a  break  point  for  dividing  the  task  of  aerial 
refueling  into  an  approach  segment  and  a  station  keeping  segment. 

A  necessary  condition  in  dividing  tasks  for  separate  practice  is 
that  any  resultant  task  grouping  make  good  sense!  It  must  bear  a 
relationship  to  overall  mission  performance  that  is  clear  to  the  student, 
or  that  can  be  made  clear  through  explanations.  Groupings  that  do  not 
meet  this  condition,  regardless  of  other  criteria,  could  lead  to 
development  of  "skills"  of  little  value  in  the  aircraft  or  that  would  be 
difficult  to  use  in  combination  with  other  skills  to  achieve  mission 
goals. 

Integrating  Tasks  Learned  Separately 

Tasks  learned  separately  must  eventually  be  integrated  into  larger 
tasks  involving  longer  sequences  of  actions  or  simultaneously  occurring 
actions.  Such  integration  will  eventually  occur  simply  if  the  larger 
task  is  practiced  as  a  whole  after  the  separate  parts  have  been  learned. 
The  integration  of  separately  learned  tasks  will  be  made  easier,  however, 
if  provisions  for  their  integration  are  made  when  the  separate  tasks  are 
being  practiced.  Such  provisions  are  described  below. 

Integrating  Sequential  Tasks 


For  separately  learned  tasks  that  normally  occur  successively  in 
time,  integrating  tasks  involves  linking  them  to  form  components  of  a 
contiguous  sequence  of  tasks.  In  such  linked  sequences,  the  ending  of 
one  task  becomes  the  cue  for  beginning  the  next.  Therefore,  practice  of 
the  first  task  should  include  some  cues  and  responses  that  will  initiate 
the  task  or  series  of  tasks  which  must  eventually  follow  it.  These 
overlapping  cues  and  responses  constitute  a  “bridge"  connecting 
sequential  tasks  that  makes  their  integration  easier  to  learn  even  though 
the  tasks  themselves  were  first  learned  separately. 

For  example,  during  early  training  a  student  pilot  having  difficulty 
learning  to  maintain  the  proper  rate  of  descent  during  a  visual  approach 


could  be  provided  additional  practice  on  just  this  segment  of  the 
approach.  When  practicing  just  the  final  approach  segment  in  an  ATD, 
however,  the  practice  should  begin  with  cues  signaling  the  turn  onto  the 
base  leg  which  immediately  precedes  entry  into  the  glideslope.  Thus,  the 
overlapping  cues  would  establish  a  bridge  linking  earlier  portions  of  the 
task  with  the  one  being  practiced. 

When  such  bridges  linking  separately  practiced  tasks  have  not  been 
built  during  initial  task  practice,  they  can  be  added.  Well-learned 
sequences  of  tasks  can  be  integrated  to  form  purposeful  mission 
activities  by  adding  during  further  practice  the  cue  that  initiates  one 
sequence  into  the  final  part  of  a  segment  that  is  to  precede  it.  Thus, 
the  initial  activity  of  a  dive  bomb  task,  transition  from  level  flight  to 
a  roll-in,  can  be  added  to  a  visual  navigation  task  in  order  to  bridge 
the  sequential  tasks  of  visual  navigation  and  dive  bombing. 

Verbal  cues  can  be  particularly  useful  in  building  bridges  to 
integrate  such  sequential  tasks.  For  example,  a  visually  detected  roll- 
in  point  for  an  intended  target  can  be  the  final  cue  in  a  visual 
navigation  task.  Through  verbal  instructions  to  initiate  an  attack  as 
soon  as  the  roll-in  point  is  reached,  the  identification  of  this  point 
can  easily  serve  as  the  initial  cue  for  the  roll-in  on  target.  The 
verbal  cue  associated  with  having  identified  the  intended  roll-in  point 
is  all  that  is  needed  to  make  the  navigation  and  dive  sequences  a  single 
integrated  task  that,  once  practiced,  can  be  easily  recalled  in  its 
integrated  form. 

Integrating  Simultaneous  Tasks 


Similarly,  tasks  that  must  be  performed  concurrently  with  .other 
tasks,  when  practiced  separately,  should  be  learned  in  a  way  that  helps 
bridge  the  isolation  of  the  separate  tasks.  Weapons  management  and 
aircraft  control  during  aerial  combat  are  examples  of  tasks  that  are 
usually  learned  separately  but  must  be  performed  concurrently  during  a 
combat  mission.  In  interdependent  tasks,  cues  pertinent  to  one  may  also 
govern  responses  normally  associated  with  the  other.  In  such  cases,  it 
is  especially  important  that  cues  critical  for  one  task  be  evident  while 
practicing  the  other,  thus  again  providing  a  link  between  the  separately 
practiced  tasks. 

For  example,  sighting  an  enemy  aircraft  while  on  a  reconnaissance 
mission  is  a  cue  to  initiating  offensive  action  and  to  selecting  and 
arming  the  aircraft's  guns.  The  two  tasks,  (1)  initiating  offensive 
action  and  (2)  selecting  and  arming  the  guns,  normally  are  practiced 
separately.  While  task  (1)  is  being  practiced,  however,  a  verbal 
response  such  as  "select  guns"  can  be  practiced  at  the  time  such  action 
should  occur  relative  to  sighting  the  enemy  aircraft  and  initiating  the 
attack  maneuver.  Separately,  while  task  (2)  is  being  practiced,  the 
words  "select  guns"  can  be  employed  as  the  cue  to  accompany  initiation  of 
the  procedural  task.  With  the  cue  to  initiation  of  task  (2)  thus 
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incorporated  into  practice  of  task  (1),  integration  of  the  two  separately 
practiced  tasks  into  a  single  mission  task  will  occur  more  easily. 


DURATION  AND  FREQUENCY  OF  PRACTICE 

Three  questions  were  frequently  asiced  by  persons  interviewed  during 
visits  to  aircrew  training  sites:  How  long  should  ATD  practice  sessions 
be?  How  long  should  a  given  task  be  practiced  during  a  single  session? 
How  frequently  should  sessions  occur?  These  questions  cannot  be  answered 
categorically.  Optimum  practice  schedules  depend  on  a  number  of  factors 
that  vary  with  the  tasks  being  trained,  when  they  occur  during  the 
training  program,  and  the  relative  skill  of  the  aircrew  students.  The 
factors  of  primary  concern  in  answering  these  questions  are:  (1)  effects 
of  Interference;  (2)  level  of  previous  learning;  and  (3)  amount  of 
forgetting  during  training. 

Effects  of  Interference 


The  factor  of  most  concern  is  interference.  When  learning  a  complex 
skill,  lack  of  mastery  of  any  one  component  of  that  skill  may  interfere 
with  correct  performance  of  anotner  if  they  are  practiced  together.  The 
result  can  be  that  neither  component  is  practiced  properly.  Conditions 
under  which  tasks  are  usually  performed,  if  they  are  included  in  the 
learning  situation,  can  also  be  distracting  when  the  tasks  are  being 
learned.  As  far  as  practice  schedules  are  concerned,  the  issue  is  how 
practice  can  be  arranged  to  avoid  interference  during  early  training  when 
It  would  be  detrimental. 

Interference  should  be  avoided  when  it  confuses  the  identification 
of  cues  to  be  learned,  or  when  it  leads  to  unmanageable  conflicts  in  the 

nature  and  magnitudes  of  responses.  Such  confusion  and  conflicts  are 

Indicated  when  progress  during  practice  ceases.  For  example,  in  low 
altitude  maneuvering  a  pilot  must  maintain  a  proper  altitude  above  the 
ground  while  simultaneously  keying  on  features  of  the  terrain  for 
navigation  purposes.  If  it  becomes  evident  during  ATD  practice  that 

little  or  no  progress  is  being  made  in  coordinating  these  activities, 

altitude  can  be  "frozen"  (If  the  design  of  the  ATD  simulation  permits 
individual  parameters  to  be  frozen)  so  the  pilot  can  practice  responding 
to  terrain  navigational  cues  without  having  to  attend  also  to  altitude 
control . 

How  long  altitude  should  remain  frozen  depends  on  two  conditions. 
First,  some  minimum  of  navigational  skill  should  be  evident.  Second, 
because  it  would  usually  be  desirable  to  practice  both  aspects  of  the 
skill  as  soon  as  progress  can  be  made  in  both,  the  instructor  should 
reactivate  the  altitude  parameter  when  sufficent  progress  occurs  in 
navigation.  If  progress  can  later  be  made  while  practicing  both  skills, 
then  joint  practice  should  continue.  If  not,  brief  practice  on  altitude 
maintenance  alone  should  be  tried. 
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In  a  number  of  instances,  two  tasks  may  be  learned  separately  to  a 
fairly  high  proficiency  level,  such  as  interpreting  a  radar  track  of  a 
hostile  aircraft  and  communicating  with  the  ground.  When  they  are 
practiced  together  for  the  first  time,  interference  is  likely,  and  some 
decrement  in  the  performance  of  both  can  be  expected.  However,  the  thing 
to  be  learned  during  such  practice  is  the  integration  of  the  two  tasks, 
and  as  long  as  the  integration  progresses,  practice  is  productive.  But 
if  performance  on  the  integrated  tasks  becomes  poorer  as  practice 
continues,  then  practice  should  cease  temporarily. 

Whether  the  break  should  be  five  minutes,  an  hour,  a  day,  or  several 
days  depends  v.n  when  the  interference  can  be  overcome.  Experience  with 
particular  training  problems  must  be  the  guide  for  determining  the  length 
of  the  break.  If  experience  indicates  that  the  confusion  is  transitory, 
take  a  five-minute  break  and  try  again.  Relaxation  during  the  break  is 
desirable,  rather  than  mental  rehearsal  of  the  problem  task.  If  a 
training  matter  must  be  discussed,  a  topic  unrelated  to  the  task  just 
practiced  should  be  chosen.  After  the  break,  the  task  at  hand  should  be 
reviewed,  focusing  upon  the  particular  discriminations  that  are  needed. 
The  student  should  be  aided  in  analyzing  his  problem,  and  provided 
guidance  (including  demonstrations  if  the  instructor  thinks  they  will 
help).  If  performance  continues  to  deteriorate  when  practice  is  resumed, 
practice  on  that  task  should  stop  for  the  time  being,  but  plans  should  be 
made  for  its  resumption  later. 

If  practice  on  a  particular  task  must  stop  because  of  interference, 
but  additional  training  time  is  available,  another  training  task  should 
be  practiced.  Usually,  the  best  choice  of  an  alternative  task  would  be 
one  that  the  student  has  previously  done  well,  but  for  which  more 
practice  is  needed.  His  previous  (partial)  mastery  of  the  .al ternate  task 
will  reduce  the  likelihood  of  interference  with  (and  from)  the  one  just 
discontinued.  He  also  will  have  a  greater  opportunity  to  be  successful 
on  the  alternative  task  because  of  his  earlier  success  with  it,  thus 
preventing  unnecessary  discouragement  (see  Chapter  VII).  A  large  number 
of  alternative  tasks  are  available  in  continuation  training  where 
maintenance  of  previously  learned  skills  is  a  major  concern.  During 
earlier  training  there  are  usually  few  choices  of  tasks  on  which  students 
have  relative  proficiency,  but  knowledge  of  a  student's  training  history 
in  relation  to  present  training  objectives  would  permit  suitable 
alternative  tasks  to  be  selected.  Emergency  procedures  that  require 
periodic  practice  for  maintenance  are  always  likely  candidates,  of 
course,  but  so  are  most  skills  that  are  taught  earlier  in  the  training 
program.  If  a  task  must  be  selected  that  has  not  been  performed 
successfully  in  the  past,  it  should  be  one  that  is  as  different  as 
possible  from  the  one  discontinued  with  respect  to  the  types  of  cue  and 
response  discriminations  required.  For  example,  if  practice  was  stopped 
on  a  flight  control  task,  the  next  one  should  be  a  procedural  task,  or 
one  involving  voice  communication,  radar  interpretation,  weapons 
management,  etc. 
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For  the  task  that  was  interrupted,  preparation  should  be  made  for  a 
later  return  to  it.  The  interference  that  caused  progress  to  cease  will 
dissipate  on  its  own  over  time,  provided  practice  was  not  continued  so 
long  that  improper  discriminations  were  learned.  However,  to  aid  in  the 
dissipation  and  to  guard  against  a  similar  problem  next  time,  the  student 
should  be  encouraged  during  the  interim  to  analyze  his  difficulties 
verbally  and  to  practice  the  task,  slowly  and  deliberately,  in  his 
imagination.  Further  practice  on  the  interrupted  task  should  not  occur 
until,  as  experience  has  shown,  the  student  can  be  expected  to  resume 
progress  toward  its  mastery. 

How  schedules  of  ATD  practice  can  be  adjusted  to  assure  that 
interference  does  not  have  a  detrimental  effect  upon  skill  development 
depends  upon  the  sources  of  the  interference.  There  are  two  major 
sources. 

Inter-Task  Competition 

Learning  a  new  skill  can  interfere  with  the  retention  and 
performance  of  a  previously  learned  one,  and  learning  one  task  can 
interfere  with  acquiring  a  subsequent  one.  Interference  is  greatest  in 
either  case  during  earlier  practice  trials  when  the  cue-response 
discriminations  are  being  learned;  but  even  during  later  stages  of 
training,  partially  learned  discriminations  may  be  required  more  rapidly 
than  the  student  can  make  them.  Interference  will  not  occur,  however,  to 
the  extent  that  the  cues  and  responses  corresponding  to  each  task  can  be 
clearly  discriminated  from  each  other. 

Interference  is  greatest  when  cue  and  response  patterns  for  two 
tasks  are  similar,  but  the  requirements  for  their  use  are  different.  For 
example,  flight  controls  at  fast  and  slow  airspeeds  feel  different  in 
many  aircraft.  If  early  ATD  training  sessions  include  both  airspeeds, 
flight  control  "feel"  at  one  speed  becomes  confused  with  that  of  the 
other.  By  gaining  an  intermediate  level  of  proficiency  at  recognizing 
and  responding  to  "feel"  cues  for  fast  speeds  first,  the  "feel"  at  low 
speeds  can  be  more  easily  recognized  as  predictably  different  from  what 
is  experienced  at  fast  speeds.  Once  the  difference  can  be  discriminated, 
differential  responses  can  be  learned  as  well. 

For  two  tasks  with  similar  patterns  of  cue-response  discriminations, 
practice  should  be  scheduled  so  as  to  permit  a  minimum  degree  of 
proficiency  to  be  developed  on  one  before  the  other  is  attempted.  If 
such  separation  is  not  possible,  they  should  be  practiced  with  enough 
time  between  them,  e.g.,  on  different  days,  so  that  the  tasks  themselves 
will  not  be  confused  with  each  other. 

Many  instances  of  inter-task  interference  during  ATD  training  were 
reported  in  the  interview  conducted  during  the  present  project.  Most 
frequently,  the  interference  involved  emergency  procedures  training. 
Malfunctions  involving  degraded  aircraft  control  were  introduced,  for 


74 


example,  before  the  student  had  developed  adequate  skills  for  aircraft 
control  under  normal  conditions.  Some  temporary  disruption  in  progress 
toward  mastery  of  aircraft  control  skills  almost  surely  resulted, 
ultimately  increasing  the  amount  of  time  and  effort  required  to  master 
the  tasks  of  controlling  the  aircraft  under  both  conditions. 

The  risk  of  deterioration  in  performance  increases  as  the  number  of 
incompletely  mastered  tasks  included  in  a  practice  session  increases. 
This  risk  should  be  anticipated  during  the  analyses  that  precede  training 
program  design  so  that  deterioration  can  be  minimized.  Appropriate 
sequencing  of  tasks  to  be  practiced  could  avoid  the  problem  by  and  large, 
but  the  syllabus  should  also  be  flexible  enough  for  instructors  to  reduce 
the  number  of  tasks  to  be  practiced  at  a  particular  time  when  performance 
deterioration  is  evident.  The  instructor  could  then  introduce  additional 
tasks  when  progress  toward  their  mastery  can  continue. 

Boredom  and  Fatigue 


If  practice  continues  when  the  student  becomes  bored  or  fatigued, 
the  student  will  be  practicing  the  behavior  of  not  attending  fully  to  the 
task.  Discriminations  will  become  lax,  and  progress  toward  skill 
development  will  suffer.  Some  instructors  insist  that  this  problem 
should  not  be  a  concern,  that  aircrew  performance  is  often  boring  and 
fatiguing,  "so  the  students  may  as  well  get  used  to  it."  In  fact,  some 
transport  and  bomber  type  ATD  training  sessions  were  noted  to  have  been 
scheduled  deliberately  for  long  time  periods  in  part  to  induce  fatigue 
and  boredom  for  that  purpose. 

It  is  important  that  aircrews  learn  to  accommodate  boredom  and 
fatigue  while  maintaining  high  levels  of  skill  performance.  However,  as 
with  other  interferences,  their  effects  can  only  be  detrimental  when 
skills  are  being  acquired.  These  or  any  other  sources  of  interference 
should  not  be  allowed  to  block  rapid  progress,  or,  because  of  student 
laxness,  result  in  learning  imprecise  skills. 

Onsets  of  fatigue  and  boredom  vary  with  training  tasks  and  types  of 
ATDs.  Instructor  judgments  are  the  best  bases  for  deciding  how  long,  and 
how  often,  various  tasks  can  be  practiced  without  risk  of  boredom  and 
fatigue.  Practice  should  be  scheduled  according  to  such  decisions. 
However,  even  when  lengthy  ATD  training  schedules  are  necessary,  both 
boredom  and  fatigue  can  be  prevented  simply  by  shifting  the  tasks  being 
practiced.  Switching  from  procedural  to  control  tasks,  from  vectoring  to 
gunnery,  from  instrument  flight  to  aerobatics,  etc.,  can  result  in 
renewed  enthusiasm,  especially  when  the  second  task  is  both  more 
interesting  and  more  challenging  to  the  student. 

Level  of  Previous  Mastery 


In  their  reviews  of  studies  on  skill  retention,  Prophet  (1976)  and 
Schendel,  Shields,  and  Katz  (1978)  concluded  that  original  level  of 
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learning  was  the  primary  determiner  of  wnat  u id  be  remembered.  To 
assure  that  a  skill  required  on  an  operational  mission  will  be  recalled, 
the  original  level  of  learning  therefore  should  be  high.  It  is  not 
sufficient  to  be  able  to  perform  a  task  concctly  once,  or  even  a  few 
times,  and  then  cease  its  practice  and  expect  to  able  to  use  it  on  a 
subsequent  mission.  Rather,  skills  must  be  "overlearned, "  that  is, 
practiced  well  beyond  the  time  proficiency  is  first  achieved.  A  rule  of 
thumb  in  deciding  how  much  practice  should  occur  for  mission  tasks  is  to 
continue  practice  until,  as  past  experience  has  shown,  correct 
performance  can  be  assured  the  next  time  that  task  is  performed. 

The  more  thoroughly  the  tasks  have  been  learned,  the  less  the 
interference  among  them.  Therefore,  as  mastery  increases,  finer  and 
finer  discriminations  of  cues  and  responses  can  be  practiced  with  less 
danger  of  previously  learned  discriminations  becoming  confused  with  new 
ones.  Also,  as  pointed  out  in  fhapter  III,  discriminations  are  built  on 
discriminations.  So,  not  only  is  deterioration  of  skills  during  practice 
less  likely  af oer  some  discriminations  have  been  learned,  but  transitory 
cessations  of  progress  will  be  rarer  as  well,  because  the  earlier  skills 
can  facilitate  the  learning  of  later  ones.  A  pilot  can  learn  more  easily 
to  maintain  altitude  above  th..  ground  when  contour  flying  over  varying 
terrain  if  he  has  first  learned  .oth  to  judge  altitude  at  low  levels  over 
flat  ground  surfaces  and  to  fly  those  maneuvers  which  permit  him  to  cross 
mountain  ridges  without  "highlighting”  himself.  These  earlier  skills 
would  facilitate,  not  interfere  v- i th,  low  level  altitude  maintenance 
because  they  would  be  incorporated  directly  into  the  newer  skill. 

After  at  least  an  intermediate  level  of  mastery  has  been  attained  on 
some  tasks,  '■elatively  longer  practice  sessions  will  be  possible  without 
causing  Interference  among  tasks.  So,  as  mastery  increases,  schedules 
for  practice  can  be  based  on  considerations  other  than  interference. 
Within  the  limits  of  student  fatigue  and  administrative  efficiency,  later 
ATD  practice  sessions  should  be  as  long  as  needed  to  ensure  both 
meaningful  task  performance  and  session-to-session  improvement  in 
performance.  Generally,  as  long  as  progress  is  maintained,  total 
practice  time  during  skill  acquisition  is  less  for  fewer,  longer  practice 
sessions  than  for  a  greater  number  of  shorter  ones. 

Amount  of  Forgetting 

The  requirement  for  scheduling  practice  to  minimize  forgetting  is  to 
provide  practice  at  such  times  tnat  the  effects  of  forgetting  can  be 
prevented  before  they  have  serious  impact  on  performance.  An  apparent 
failure  to  consider  the  need  for  continued  practice  to  prevent  forgetting 
was  noted  at  one  training  site  during  the  present  project.  Instructors 
complained  that  students  who  had  achieved  proficiency  in  maneuvers 
practiced  early  in  ATD  training  had  "forgotten"  the  skills  by  the  time 
they  reached  an  aircraft.  Occasional  practice  of  the  tasks  involved, 
spaced  throughout  ATD  training,  would  have  prevented  the  difficulty. 


Thus,  practice  schedule  requirements  for  minimizing  forgetting  may 
differ  from  those  for  original  skill  acquisition.  The  reason  for  this 
difference  is  that  skills  will  be  retained  and  ready  for  use  to  the 
extent  that  their  cue-response  patterns  can  be  associated  with  the 
circumstances  in  which  they  are  needed.  Circumstances  vary  with  time: 
Different  types  of  maneuvers  are  flown;  flights  around  or  to  new  field 
locations  introduce  new  terrain  conditions  and  cues;  new  skills  are 
learned,  as  are  new  integrations  of  old  skills;  new  experiences 
accumulate  with  time,  and  responsibilities  of  aircrews  change  with 
professional  maturity.  Effects  of  those  occurrences  become  superimposed 
so  to  speak  on  previous  knowledge  and  skills,  causing  them  to  lose  some 
of  their  original  precision.  Therefore,  the  skills  to  be  maintained  must 
be  practiced  under  similarly  varying  times  and  circumstances  so  that 
later  variations  in  cues  and  responses  will  not  interfere  with  their 
performance. 

Implication  for  Scheduli ng  Training 

Answers  to  the  three  questions  asked  at  the  beginning  of  this 
section  can  now  be  viewed  in  terms  of  the  factors  addressed  above,  i.e., 
the  influences  of  interference  during  practice,  the  level  of  learning 
previously  achieved,  and  the  amount  of  forgetting  during  training. 
Whether  the  concern  is  the  length  of  a  single  practice  session,  the 
amount  of  practice  on  a  given  task  within  a  practice  session,  or  the 
length  of  time  between  practice  sessions,  there  are  three  rule-of-thumb 
criteria  for  decisions  affecting  schedules.  These  criteria  are:  (1)  The 
time  between  occasions  for  practice  should  not  be  long  enough  for 
noticeable  performance  decrements  to  occur.  (2)  Since  the  effects  of 
interference  are  greatest  among  incompletely  learned  tasks,  frequent  but 
relatively  short  practice  sessions  would  be  best  for  novices.  (3)  When 
the  occasion  for  refresher  training  arises,  especially  with  highly 
experienced  aircrews,  concentrated  practice  within  relatively  long 
individual  sessions  would  probably  be  more  efficient  than  practice 
distributed  over  a  greater  number  of  rapidly  recurring  short  sessions. 


SUMMARY 

Five  topics  of  concern  in  structuring  ATD  training  were  discussed  in 
this  chapter:  establishing  priorities  for  ATD  use;  sequencing  ATD 

training  relative  to  academic  and  flight  training;  allocating  training 
among  various  types  of  available  ATDs;  organizing  tasks  into  practice 
sessions;  and  determining  the  frequency  and  duration  of  individual  ATD 
training  sessions.  The  major  points  in  these  discussions  are  summarized 
below. 

•  Priorities  for  scheduling  ATD  use  should  be  developed 
systematically,  considering  the  needs  for  serving  different  groups  as 
well  as  relative  needs  within  each  group.  Priorities  for  training  groups 
such  as  combat  crews,  instructors7  maintenance  personnel,  etc.,  should  be 
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defined,  as  should  priorities  for  types  of  tasks  such  as  aircraft 
control,  emergency  procedures,  low  level  navigation,  etc.  These 
priorities  should  govern  ATD  use  as  Its  availability  changes  as  well  as 
during  day-to-day  training. 

•  ATD  training  priorities  should  optimize  the  effectiveness  and 
efficiency  of  all  available  devices.  Factors  to  consider  are  the  kinds 
of  training  for  which  individual  ATDs  are  best  suited,  relative 
effectiveness  and  efficiency  of  devices  for  training  given  tasks,  cost  of 
training,  and  availability  of  alternative  media,  including  aircraft. 

•  Course  syllabi  should  reflect  training  priorities  and  identify 

alternative  resources.  If  an  ATD  cannot  be  available  for  training  a 
given  group  or  task  because  higher  priority  needs  must  be  met, 
instructional  personnel  should  be  aware  of  alternative  resources  and 
their  relative  merits. 

t  ATD  practice  should  be  designed  and  scheduled  to  provide 

experience  in  the  use  of  knowledge  and  concepts  previously  learned  at  the 
verbal  level  in  academic  training.  Verbal  knowledge  must  be  related  to 
concrete”  experiences  if  it  is  to  have  operational  meaning.  ATDs  should 
be  used  to  provide  such  experiences  when  feasible. 

•  When  students  have  not  had  experiences  needed  for  academic 

concepts  to  be  understood,  ATD  experience  of  an  appropriate  kind  should 
precede  or  be  concurrent  with  academic  training  in  those  concepts. 
Verbal  descriptions  of  ground  effects,  for  example,  arid  related 

coordinated  control  adjustments  can  have  only  limited  meaning  at  best  to 
a  student  who  has  never  experienced  ground  effects.  Experience  with 
simulated  ground  effects  (or  actual  ones  in  aircraft)  will  enable  the 
student  in  his  imagination  to  integrate  control  adjustments  with 
conditions  under  which  they  occur. 

•  ATD  and  aircraft  training  should  be  sequenced  as  needed  to 
maxi mi ze  the  contributions  of  both.  No  current  ATD  has  perfect  fidelity, 
nor  can  Tf  simulate  ffie  full  range  of  aircraft  cues  and  reactions. 
Because  of  incompleteness  of  simulation,  the  full  meaning  of  some  ATD 
experiences  cannot  be  understood  until  after  similar  tasks  have  been 
attempted  in  an  aircraft.  Rapid  learning  of  discriminations  of 
transferable  cues  and  responses  will  often  depend  on  timely  experiences 
of  similarities  and  differences  between  the  aircraft  and  ATD. 

t  The  experience  level  of  a  student  should  help  determine  how  long 
are  delays  among  related  academic,  ATD,  and  aircraft  experiences.  An 
experienced  aircrewman  has  a  repertoire  of  meanings  and  discriminations 
that  enable  him  to  derive  more  meaning  than  a  novice  can  from  most  types 
of  training  experiences.  He  can  also  retain  what  he  learns  longer. 
Thus,  timing  of  particular  academic,  ATD,  and  aircraft  training  can  be 
more  flexible  for  an  experienced  aircrewman. 
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•  For  any  task  trained  in  an  ATP,  key  cues  and  responses  related  to 
the  task  should  be  represented  meaningfully.  To  the  extent  such  cues  and 
responses  can  be  visualized  by  the  student,  strict  device  fidelity  is  not 
needed.  When  cues  and  responses  cannot  be  clearly  imagined,  training 
should  be  allocated  only  to  ATDs  with  a  necessary  minimum  of  cue  and 
response  realism. 

•  If  precise  visual -motor  skills  are  to  be  learned,  precise 
performance  must  be  practiced.  When  skills  to  be  learned  involve 
coordination  of  cue-response  systems  based  on  realistic  kinesthetic 
and/or  visual  feedback,  the  ATD  should  have  realistic  control  dynamics 
and  provide  realistic  task-intrinsic  feedback. 

•  The  experience  levels  of  students  should  be  considered  when 
allocating  training  to  ATDs  with  differing  capabilities.  Fidelity  of  a 
given  device  is  not  as  important  for  training  an  experienced  aircrewman 
to  perform  a  given  task  as  for  a  novice.  In  his  imagination,  an 
experienced  aircrewman  can  compensate  for  lack  of  fidelity.  He  can  "see" 
an  instrument  change,  "hear"  an  engine  start,  or  "feel"  an  acceleration. 

•  Training  should  be  allocated  to  individual  ATDs  according  to 
their  effectiveness  and/or  efficiency  in  training  portions  of  tasks. 
When  additional  practice  Is  needed  For  some  parts  of  a  task  fe.g. , 
vectoring  an  aerial  intercept),  but  not  for  other  parts  (piloting  the 
aircraft),  similar  training  effectiveness  and  greater  overall  efficiency 
can  be  achieved  if  the  portion  requiring  the  greater  amount  of  effort  is 
practiced  in  a  part- task  trainer,  thus  freeing  more  comprehensive  devices 
for  other  purposes. 

•  Training  can  be  allocated  to  an  ATD  with  characteristics 
different  from  a  particular  target  aircraft  provided  that  the  skills 
being  practiced  are  not  specific  to  the  aircraft.  B-52  and  C-135  pilots 
can  profitably  practice  instrument  fl  ight  skills  in  a  T-40  trainer 
because  the  key  aspects  of  these  skills  do  not  depend  upon  cockpit 
configurations,  and  very  little  upon  the  dynamics  of  particular  aircraft. 

•  When  two  or  more  ATDs  are  equally  useful  for  training  a 
particular  set  of  skills,  allocation  of  training  to  a  device  should 
consider  relative  cost,  overall  program  efficiency,  and  the  preferences 
of  the  instructors  and  students.  Operation  and  support  costs  vary  among 
ATDs,  and  use  of  part- task  trainers  can  free  more  comprehensive  devices 
for  comprehensive  training.  Both  considerations  are  important.  However, 
the  preferences  (usually  for  high  fidelity  devices)  of  instructors  should 
be  considered  as  well.  Being  forced  to  use  less  desirable  ATDs, 
especially  when  a  preferred  one  remains  idle,  will  likely  have  a  negative 
effect  on  user  attitudes. 

•  Tasks  should  be  broken  down,  or  grouped,  for  practice  so  as  to 
reduce  cue/response  complexity  early  in  training,  or  to  provide 
opportunities  to  accommodate  complexity  later  in  training.  Efficient 
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training  often  requires  reduced  task  complexity  to  avoid  interference 
while  learning  basic  discriminations.  However,  an  aircrewman  must 
eventually  learn  to  accommodate  all  interferences  characteristic  of 
operational  performance. 

•  Complex  tasks  should  be  broken  down  into  parts,  or  subtasks,  if 
it  makes  the  parts  easier  to  learn.  Radio  c omnium c a t i off,  T br  ex ample ~ 
can  be  learned  more  "easily  if  practiced  separately  from  early  attempts  at 
ai rcraft  control . 

•  Tasks  should  be  separated  from  each  other  during  early  learning  { 

if  such  aP~separati  on  aids  cue  discernment.  The  recognition  of  aural  cues  1 

i  ndicatmg"  engine  maTperTonnance  can" be  difficult  to  learn  if  there  are  j 

other  unusual  sound  cues  present  as  well. 

•  Tasks  separated  for  practice  should  be  divided  so  as  to  maintain 
integrity  of  the  parts.  An  approach  to  a  tanker  for  refueling  can  be 
practiced  separately  from  station  keeping  after  getting  into  position. 

•  Whatever  separations  of  tasks  are  made, _ the  resul ting  groupi ng 

should  make  sense  to _ the  student.  He  should  be  able  to  recognize  the 

reTations  Between  the~tasks  practiced  and  overall  mission  performance. 

•  When  tasks  normally  occur^  together  or  in  a  contiguous  sequence, 
but  are  practiced  separately,  some  cues/responses  rel ated  to  the  omi tied 
tasks  should  appear  at  appropn¥te~Times  duriruT pr actTce .  The  purpose  i  s 
to  avoid  isolation  of  the  separate  but  interdependent  tasks.  By 
responding,  at  least  symbolically,  to  real  or  imagined  cues  related  to 
the  omitted  tasks,  overall  task  awareness  can  be  maintained,  and  the 
eventual  integration  of  tasks  enhanced. 

•  ATP  practice  sessions  should  be  of  a  duration  and  frequency  that 
permit  steady  progress~~during  learning.  Too^Tong  and  too  frequent 
practice  sessions  early  in  Teaming  result  in  interference  among  cues  and 
responses  within  tasks,  and  from  one  task  to  another.  Practice  should 
cease,  or  subsequent  sessions  be  delayed,  when  progress  ceases. 

•  Schedules  for  practice  should  prevent  inter- task  interference  as 
much  as  possible.  If  two  tasks  are  likely  to  become  conTused  with  each 
other  IT  practiced  together  originally,  either  partial  mastery  on  one 
task  should  be  achieved  prior  to  beginning  the  other,  or  practice  on  the 
separate  tasks  should  be  at  different  times,  preferably  on  different 
days. 

•  Practice  during  task  acquisition  should  not  continue  to  the  point 
that  a  student  becomes  unduly  bored  or  fatiguedT  If  students  practice 
1 ackadai sical  performance,  they  learn  lackabaTsical  performance.  Both 
boredom  and  fatigue  can  often  be  alleviated,  however,  by  switching  to  a 
new  task,  especially  if  it  is  more  interesting  and  challenging. 
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•  As  skill  mastery  progresses, practice  sessions  can  be  both  J_onger 
and  more  frequent.  Interference  is  less  and  less  a- problem  as~cue  and 
response  di scr iminations  are  learned. 

•  p  rac  tice _ sessions  should  be  frequent  enough  to  prevent 

unacceptabl e  deterioration  of~ sTclTls.  This  hoi ds  for  skTTT  acquTsitTo'n 
and  For  proTTcTency  maintenance .  Experience  with  retention  of  individual 
skills  should  be  the  basis  for  scheduling  repetitive  practice  during 
training  and  for  skills  maintenance  following  training. 
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CHAPTER  VI 


ATD  INSTRUCTORS 


The  notion  persists  that  the  characteristics  and  behavior  of 
instructors  are  important  factors  influencing  aircrew  training 
effectiveness.  Reviews  of  the  literature  on  the  flight  instructor 
(Smode,  Hall,  and  Meyer,  1967)  and  of  ATD  instructor  utilization 
practices  of  the  Air  Force  (Caro,  1977)  have  ; fai 1 ed  to  find  empirical 
support  for  this  notion.  Nevertheless,  it  was  observed  during  the 
present  survey  that  there  tended  to  be  differences  among  the  prestige, 
the  role,  and  the  qualifications  of  ATD  i  instructors,  and  that, 
intuitively  at  least,  these  instructor  factors  seemed  related  to  the 
effectiveness  of  various  ATD  training  activities.  Specifically,  In  the 
ATD  training  activities  judged  to  be  more  effective, 1  (1)  ATD  instructors 
are  viewed  as  having  special  capabilities,  /( 2)  they  are  employed  as 
flight  instructors  as  well  as  ATD  instructors^  and  (3)  they  are  provided 
extensive  training  to  qualify  them  for  their  i/se  of  ATDs. 


ATD  INSTRUCTOR  PRESTIGE 

Three  factors  were  identified  that  could  be  characterized  as 
influencing  the  prestige  of  ATD  instructors.  These  factors  are:  (I)  how 
the  instructor  was  selected;  (2)  the  priority  given  to  his  instructional 
duties  vis-a-vis  other  unit  duties;  and  (3)  the  kinds  of  assignments  he 
would  receive  following  a  tour  as  an  ATD  instructor. 

Instructor  Selection 


The  ATD  training  activities  judged  most  effective  were  those  in 
which  the  instructors  were  selected  from  among  volunteers  who  sought  the 
assignment.  The  selectees  typically  were  aircrewmen  who  had  completed 
one  but  not  more  than  two  full  tours  in  operational  units  and  were  judged 
by  themselves  as  well  as  by  their  peers  to  have  performed  well  above 
average  in  those  assignments.  They  were  Captains  and  junior  Majors,  and 
they  were  anticipating  normal  career  progression  and  a  career  in  the 
service. 

The  selection  of  ATD  instructors  in  the  more  effective  programs  was 
made  by  personnel  from  the  training  unit  on  the  basis  of  personal 
knowledge  of  the  qualifications  of  the  selectee  and  judgments  as  to  the 
suitability  of  his  personality.  Personnel  were  sought  who  exhibited 
personality  characteristics  such  as  maturity  and  stability,  believed  to 
be  conducive  to  the  maintenance  of  favorable  working  relationships  with 


*The  basis  for  judgments  of  the  relative  effectiveness  of  the 
ATD  training  observed  during  this  project  is  described  in  Chapter  II. 
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peers  and  students.  Knowledge  of  the  selectee  was  gained  in  part  through 
observation  of  his  performance  during  periodic  requalification  training 
or  flight  performance  checks.  Before  final  selection,  the  candidates 
were  typically  interviewed  by  personnel  from  the  training  unit.  The 
personnel  doing  the  selecting  were  the  instructors  with  and  for  whom  the 
selectee  would  be  working. 

By  contrast  with  the  above,  ATD  instructors  in  the  less  effectiv  • 
programs  were  more  likely  to  have  been  selected  by  an  impersonal  process, 
not  to  have  volunteered  for  the  assignment,  and  to  have  never  been 
assigned  to  operational  units  as  aircrewmen.  They  tended  to  be  of 
slightly  lower  rank  (e.q.,  junior  Captains),  although  some  were  more 
senior  and  were  on  a  "twilight  tour."  They  also  tended  to  have  spent  a 
disproportionate  amount  of  their  prior  service  careers  as  instructors. 

None  of  the  noted  instructor  selection  procedures  had  been 
validated,  and  no  formal  studies  are  known  to  have  established  a 
relationship  between  instructor  abilities,  experience,  or  personality  and 
ATD  or  flight  instructional  effectiveness.  Further,  the  selection 
procedures  associated  with  the  more  effective  training  programs  were 
informal  in  nature  and  may  have  been  biased  toward  factors  of  little  or 
no  real  signi ficance.  However,  there  are  reasons  to  believe  that  these 
procedures  were  beneficial.  It  is  believed  that  they  served  to  identify 
and  eliminate  marginal  pilots,  those  not  wanting  to  instruct,  and  others 
whom  the  selecting  instructor  felt  would  probably  be  poor  instructors. 
Furthermore,  the  esprit  de  corps  of  the  instructors  appeared  to  be 
increased  by  the  fact  that  they  were  members  of  a  "select"  group,  i.e., 
had  survived  a  selection  process  operated  by  respected  peers,  and  that 
their  judgments  as  professional  teachers  were  valued  as  a  selection  tool. 
(Chapter  VII  discusses  the  importance  of  instructor  attitudes  and 
professional i sm. ) 

Priority  of  Instructional  Duties 

All  military  personnel  are  subject  to  extra  duties  that  can  detract 
from  their  primary  duties.  ATD  instructors  are  no  exception.  In  the 
more  effective  ATD  training  programs  surveyed,  however,  instructional 
duties  were  given  priority.  Instructors  tended  to  have  relatively  few 

competing  additional  duties,  and  they  had  more  flexibility  in  scheduling 
their  involvement  in  those  duties  than  did  their  counterparts  in  other 
programs.  Additionally,  their  workloads  were  managed  so  that  the  demands 
made  on  them  did  not  detract  from  their  instructional  duties. 
Instructors  were  allowed  time  to  prepare  for  ATD  sessions  before  meeting 
with  the  students  in  recognition  that,  at  some  point,  they  need  to  review 
the  students'  history-strengths ,  weaknesses,  performance  problems, 
massing  training  elements--as  well  as  the  session  objectives,  performance 
r'-.eria,  priorities,  and  implementation  procedures.  Consideration  was 

'.r>own  when  instructors  conducted  ATD  training  during  normally  off- 

nours.  The  same  instructors  were  not  repeatedly  scheduled  for  this 
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duty;  and  when  they  were  scheduled  for  normally  off-auty  hours,  they  were 
given  compensatory  time  <  ff  when  possible. 

The  performance  of  AID  instructors  in  the  more  effective  programs 
was  assessed  by  training  supervisors  or  SlAh-EVAL  personnel  who  conducted 
periodic  proficiency  checks.  The  major  focus  of  these  instructor 
evaluations  was  on  the  manner  in  which  ATD  training  was  conducted  and  th^ 
relevance  of  that  training  to  t  e  course  objectives.  These  evaluations 
of  proficiency  were  believed  by  the  instructors  to  be  weigr.ed  heavily  in 
the  determination  of  their  job  proficiency  ratings. 

By  contrast,  instructors  in  the  less  effective  ATD  training  programs 
were  more  likely  to  view  their  involvement  in  those  programs  as  "just 
another  duty"  that  took  up  time  they  felt  could  be  better  spent 
elsewhere.  They  believed  their  effectiveness  was  assessed  on  the  basis 
of  their  performance  of  their  non-ATQ  related  duties.  In  fact,  they  were 
generally  unaware  of  any  systematic  efforts  by  unit  supervisors  to  assess 
their  effectiveness  as  ATD  instructors. 

The  issue  of  instructor  workloads  and  the  scheduling  of  ATD 
activities  during  off  hours  is  complicated.  On  one  hand,  schedulers  fee1 
that  high  fidelity  ATDs  are  too  expensive  to  let  sit  idle  at  night  or  on 
weekends.  On  the  other  hand,  military  personnel  frequently  have  daytime 
collateral,  or  even  primary,  duties  in  addition  to  ATD  instruction.  They 
may  therefore  be  required  to  work  during  normal  duty  hours  and  also  at 

night  or  on  weekends  when  it  is  necessary  for  access  to  an  ATD.  In  cases 
where  an  instructor  teaches  late  at  night  and  then  again  early  tne  next 
morning,  fatigue  and  performance  efficiency  will  be  a  problem.  This 

situation  is  exacerbated  by  the  fact  that  military  personnel  deploy  for 
extended  periods  of  time,  so  their  off-duty  time  is  needed  for  family 

relationships  and  ether  personal  matters.  in  the  long  run,  instructor 

effectiveness  is  likely  to  suffer  unless  duty  schedules  are  such  so  as 

not  to  make  training  students  a  burden. 

Subsequent  Assignments 

In  many  instances,  ATD  instructors  viewed  their  current  assignment 

as  a  neutral  factor  so  far  as  a  career-favorable  subsequent  assignment 

was  concerned.  In  the  more  effective  programs,  however,  instructors  felt 
that  they  were  given  c  nsideration  in  subsequent  assignments  commensurate 
with  their  skills  and  other  qualifications.  These  were  the  same 

instructors  who  had  been  selected  initially  because  of  above  average 
capabilities,  and  they  expected  that  their  capabilities  would  continue  to 
be  rewarded.  They  were  able  to  cite  examples  of  instructors  who  had 
preceded  them  who  had  experienced  such  favorable  reassignment. 

A  negative  factor  related  to  subsequent  assignment  was  pointed  out 
by  some  of  the  instructors,  however.  In  some  u.iits,  ATD  instructors  were 
grouped  for  rating  purposes.  Where  this  occurred,  instructors  reported 
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that  their  assignment  to  the  group  would  hurt  their  chances  for  a 
favorable  rating  and  subsequent  assignment.  These  aircrewmen  viewed 
themselves  as  significantly  above  average  when  compared  with  the  overall 
aircrew  population,  but  as  more  nearly  average  within  their  elite  group. 
Thus  their  rating  would  be  unreal istically--and  unfavorably--depressed. 
The  effect  of  such  groupings  for  rating  purposes  was  also  noted  to  be  a 
negative  factor  in  continuing  efforts  to  select  the  more  capable 
i nstructors. 


ATD  INSTRUCTOR  ROLE 

There  is  no  empirical  evidence  that  indicates  rated  personnel  are 
required  for  ATD  training.  As  a  general  rule,  nevertheless,  ATD 
instructors  are  flight-rated  aircrewmen  who  instruct  in  aircraft  as  well 
as  in  devices.  During  the  surveys,  a  consensus  was  noted  that  the  use  of 
rated  personnel  in  all  ATDs,  even  in  CPTs,  was  desirable,  for  these 
devices  could  then  be  used  for  a  greater  portion  of  training.  Rated 
aircrewmen  are  able  to  provide  mediational  instruction  as  described  in 
Chapter  IV  which  compensates  for  lower  device  physical  fidelity.  Thus, 
they  can  train  tasks  with  high  cognitive  loadings  such  as  crew 
coordination  and  responding  analytically  to  unusual  emergency  or  tactical 
situations.  The  belief  was  frequently  expressed  that  rated  instructors, 
in  contrast  to  nonrated  personnel,  are  often  able  to  provide  supplemental 
information  about  flying  and  airborne  system  operation,  thus  creating 
training  situations  in  all  ATDs  that  have  greater  realism  and  functional 
similarity  to  operational  tasks.  Also,  flight-rated  ATD  instructors  can 
provide  mediational  instruction  which  helps  integrate  ATD  and  aircraft 
training.  Indeed,  in  many  units,  the  same  instructors  were  found  to  be 
used  for  a  student's  ATD  and  aircraft  training. 

In  addition  to  their  increased  instructional  capabilities,  many 
units  felt  that  rated  aircrewmen  were  necessary  to  improve  the 
credibility  of  ATD  training  and  to  promote  its  acceptance.  For  example, 
given  an  emphasis  on  the  inclusion  of  only  "need-to-know"  information  for 
ATD  training,  many  units  felt  that  rated  instructors  were  necessary  to 
give  credibility  to  such  information.  Additionally,  use  of  rated 
instructors  was  viewed  as  a  manifestation  of  command  emphasis  on  ATD 
training. 

There  are  exceptions  to  the  use  of  flight-rated  personnel  to 
instruct  in  ATDs,  however.  In  the  case  of  some  older  instrument 
procedures  training  devices  employed  in  undergraduate-level  training, 
nonrated  personnel  are  employed  as  instructors.  Some  of  these  are 
civilian  employees  who  were  formerly  flight  rated  military  personnel. 
Also,  some  ATDs  are  manned  by  nonrated  device  operators  or  technicians 
who  sometimes  function  as  instructors  when  a  flight-rated  aircrewman  is 
not  available  to  conduct  training  in  the  device.  This  latter  practice 
was  generally  viewed  as  inappropriate  by  the  aircrewmen  who  had 
participated  in  such  training,  reportedly  because  the  technicians  were 
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insufficiently  knowledgeable  with  respect  to  situational  factors  that 
tended  to  arise  during  such  training.  Technicians,  for  example,  might 
unknowingly  introduce  unrealistic  compound  malfunctions  which  would 
inevitably  cause  a  crash. 

The  assumed  advantages  of  using  nonrated  personnel  are  that  training 
personnel  costs  are  lower,  demand  for  rated  personnel  and  especially 
rated  instructors--currently  a  precious  resource  in  the  military— is  not 
as  high,  and  under-utilization  of  rated  personnel  inherent  in  some 
routine  task  assignments  is  reduced.  So  far  as  could  be  determined 
during  the  current  project,  there  have  been  no  formal  studies  to 
determine  the  effectiveness  of  the  use  of  nonrated  personnel  to  instruct 
in  ATDs. 

The  ATD  training  programs  that  were  judged  most  effective  did  employ 
instructor  pilots  in  the  dual  role  of  flight  and  ATD  instructors,  and  the 
same  instructors  were  scheduled  to  conduct  both  flight  and  ATD 
instruction  for  a  given  student.  The  few  programs  surveyed  in  which  the 
roles  of  ATD  and  aircraft  instructors  were  assigned  to  separate 
individuals  were  judged  less  effective,  as  were  those  in  which  device 
operators  were  permitted  to  conduct  instruction  when  qualified  aircrewmen 
were  absent. 


ATD  INSTRUCTOR  TRAINING 

The  structure  and  content  of  ATD  instructor  training  programs  varied 
widely  among  the  activities  surveyed.  The  training  ranged  from  a 
"checkout"  on  the  device's  instructor  console  during  which  the  location 
and  function  of  controls  and  displays  were  illustrated,  to  a 
comprehensive  program  involving  training  in  instructional  technology 
(encompassing  some  of  the  information  contained  in  Chapters  III  and  IV  of 
the  present  report),  instruction  on  how  and  when  to  employ  the  device's 
instructional  features,  and  the  conduct  of  ATD  instruction  under  the 
supervision  of  a  qualified  instructor.  The  programs  that  were  judged  to 
be  effective  were  also  the  programs  in  which  the  instructor  training  was 
judged  to  be  comprehensive  and  formally  structured. 

Several  ATD  instructor  training  programs  included  in  the  survey  were 
found  to  have  features  that  one  would  expect  to  produce  well -qualified 
instructors  and,  in  turn,  effective  ATD  training  for  student  aircrews. 
One  program  conducted  for  ATD  instructor  selectees  was  judged  to  be 
particularly  good  with  respect  to  content  and  structure.  Principal 
features  of  that  program  are  described  below. 

The  program  consists  of  four  phases.  During  the  first  phase,  the 
newly  selected  instructor  goes  through  the  instructional  program  he  has 
been  selected  to  teach.  Thus,  in  the  role  of  a  student,  he  receives  all 
instruction,  including  that  given  in  the  ATDs  and  in  the  aircraft,  that 
other  students  receive.  The  purpose  of  this  phase  of  his  training  is  to 
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C  I 


Ms  own  competence  as  an  aircrewman  and  to  assure  that  his 
■  "  :  ranee  of  all  aircrew  tasks  conform  to  specified  standards. 

iorally,  the  instructor  observes  AID  instructional  techniques  as 

are  mew ed  by  a  student  and  becomes  familial  with  the  content  and 
:o  of  instruction  in  the  course  he  will  teach. 

C ! ng  the  second  phase,  the  instructor  selectee  observes  the 
a  ,1m- ;•!  ation  to  two  students  of  the  course  he  is  being  prepared  to 
t-  »•;!:.  during  this  activity,  he  observes  and  is  instructed  as  to  (1)  the 
voe  '‘tijn  of  the  ATDs,  (2)  the  use  of  instructor  job  aids  (e.g.,  lesson 
gri  grade  slips),  and  (3)  the  employment  of  ATD  instructional  support 
•at.tas  for  the  training  he  will  conduct.  Concurrently,  he  is  tutored 

individual  basis  by  a  qualified  instructional  technologist 
r-  s.-rnlng  the  principles  of  instruction  and  how  those  principles  may  be 
auMieo  to  the  training  that  he  will  conduct.  Additionally,  to  promote 
i i :  ••  acceptance  of  ATD  training,  the  instructor  receives  information  on 
tr,;  v-?.l>ie  cf  AfD  training  in  the  program  he  will  teach  and  on  the  value 
of  A.D  training  in  general. 

During  the  next  phase,  the  instructor  selectee  conducts  selected 
portions  of  the  training  program  he  is  preparing  to  teach,  with  a 
Qualified  instructor  playing  the  role  of  a  student  and  presenting  the 
i;‘tructor  trainee  with  common  student  problems.  Following  each  training 
nod,  he  and  the  "student"  review  and  critique  his  performance.  The 
■■pose  of  this  activity  is  to  ensure  that  he  knows  why  he  does  what  he 
*s,  as  well  as  how  to  do  it. 

finally,  the  instructor  selectee  is  assigned  two  incoming  students 
to  train.  This  fourth  phase  of  his  preparation  consists  of  the  training 
f  there  two  students  under  the  supervision  of  a  qualified  instructor, 
bus,  the  final  phase  of  his  training  is  supervised  "practice  teaching," 
uring  which  he  continues  to  be  critiqued  as  necessary  by  the  supervising 
instructor.  At  the  conclusion  of  this  fourth  phase,  he  is  considered  to 
te  a  ful'y  qualified  instructor. 

To  improve  the  efficiency  of  this  rather  extensive  instructor 
tuning,  proficiency  advancement  is  used  throughout  all  the  phases  of 
the  course.  Progress  in  instructional  performance,  in  addition  to 
aircrew  skills  and  knowledge,  is  used  as  the  principal  basis  for  advance¬ 
ment  through  the  course.  Additionally,  the  instructor  trainee's  workload 
is  managed  so  that  the  demands  of  collateral  duties  do  not  detract  from 
his  studies.  Finally,  commensurate  with  the  importance  placed  on 
Instructor  training,  the  job  evaluation  reports  of  the  instructor  trainee 
reflect  the  evaluation  of  his  performance  in  the  instructor  training 
course. 

It  was  also  observed  during  the  present  study  that  It  was  not  just 
the  adequacy  of  the  initial  instructor  training  that  influenced  the 
competency  of  ATD  instructors,  but  also  the  adequacy  of  the  continuation 
training  that  they  received.  In  most  units,  ATD  instructors  participated 
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in  continuation  training  that  was  designed  to  maintain  orl;  siei- 
aircrew,  and  not  instructional,  skills.  In  a  few  units,  howe* e- , 
instructors  received  regularly  scheduled  continuation  training  designed 
to  refresh,  refine,  update  and  expand  their  ATD  instruct i>»nal 
capabilities.  Training  specialists,  ISD  personnel,  or  ATD  training 
officers  were  employed  to  provide  this  training  on  an  individual  ir  ;  -o;p 
basis.  There  were  also  supervised  instructor  workshops,  seminars.  d 
bull  sessions  that  were  designed  to  encourage  instructor  commumcetic.. 
about  personal  experiences  with  various  ATD  instructional  techniques. 
Individualized  remedial  training  was  provided  to  address  deficient,  i 
identified  in  an  instructor's  performance  during  the  routine  evaluati:>' 
of  his  instructional  skills. 

There  are  no  data  to  demonstrate  conclusively  that  any  part  of  t.vi s 
instructor  training  is  critical  to  the  effectiveness  of  instructors.  It 
may  well  be  that  other  factors,  e.g.,  the  basis  for  their  selection  and 
the  status  they  are  afforded,  would  more  than  account  for  the  fact  :r.t 
the  ATD  programs  they  are  conducting  were  judged  to  be  among  the 
surveyed. 


SUMMARY 

This  chapter  discussed  three  topics  of  concern  regarding  AID 
instructors:  instructor  prestige  and  fa~tors  that  govern  it;  instructor 
roles  and  aircrew  skills  required  to  ft ?  ?  them;  and  instructor  training 
needed  for  effective  ATD  instruction.  Ma^cr  points  in  these  discussions 
are  summarized  oelow. 

•  The  selection  of  personnel  to  be  ATD  instructors  should  be  baser 

on  their  desire  and  ability to  do  the  job.  The  most  effective  ATD 

instructors  are  those  who  seek  their  assignment ,  demonstrate  a  well  above 
average  level  of  performance  during  prior  assignments  to  operational 
units,  ana  are  mature,  stable  individuals  who  work  well  with  other 
people. 

•  Instructional  duties _ of  ATD  instructors  should  be  giver 

priority.  The  workload  of  ATD  instructors  should  be  managed  so  that  the 
demands  made  on  them  do  not  detract  from  their  instructional  duties  nor 
create  unpleasant  conditions  for  the  conduct  of  those  duties.  Moreov.. *■, 
evaluation  of  instructional  proficiency  should  be  weighted  heavily  in 
overall  instructor  job  proficiency  ratings. 

•  ATD  Instructors  should  be  given  consideration  in  subsequent 
a ssignments  commensurate  with  their  s kills  and  other  qualifications.  A'i D 
instructors  should  be  treated  equitably  with  respect  to  their  peers  i 
other  billets.  If  they  receive  insufficient  rewards,  it  will  bt 
difficult  to  staff  ATD  positions  w’th  capable,  motivated  personnel. 
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•  ATP  instructors  should  be  flight  rated  crewmen  when  the 

instruction  involves  other  than  established  procedural  tasks.  The 

knowledge  gained  through  experience  as  an  aircrewman  will  contribute  to 
effectiveness  when  instruction  involves  adaptive  behavior,  cognitive 
skills,  general izable  skills,  and  factors  that  contribute  to  an  awareness 
of  overall  mission  requirements  and  activities.  Additionally,  the  use  of 
rated  personnel  permits  the  use  of  the  same  instructor  for  a  student's 
ATD  and  aircraft  training. 

•  ATD  instructor  training  should  be  comprehensive  and  formally 

structured.  It  should  assure  that  his  performance  of  aircrew  tasks 
conforms  to  specific  standards,  that  he  has  a  practical  knowledge  of 

instructional  technology,  including  how  and  when  to  use  a  device's 
instructional  features,  and  that  he  understands  the  value  of  ATD 
training. 

•  The  instructor  trainee  should  receive  practice  in  conducting 
instruction  in  the  ATD  with  supervision  and  critiques  provided  by  a 
qualified  instructor.  Practice  teaching  helps  the  instructor  trainee  to 
gain  proficiency  in  ATD  operation  and  to  become  familiar  with  desired 
training  techniques.  It  should  also  be  used  to  review  common  student 
problems,  thus  giving  the  instructor  guided  practice  in  helping  students 
solve  problems. 

•  Evaluation  reports  for  instructor  trainees  should  reflect  their 
achievements  in  the  instructor  training  course,  and  advancement  through 
instructor  training  should  be  on  a  rroficiency  basis.  Evaluation  of 
instructor  achievement  and  proficiency  should  be  based  primarily  on 
instructional  performance  but  also  on  competence  in  the  aircrew  skills 
and  knowledge  they  will  teach. 

•  Throughout  his  tenure  as  an  ATD  instructor,  the  instructor  should 
receive  continuation  training  designed  to  refresh,  refine,  update,  and 
expand  his  ATD  instructional  capabilities.  This  training  should  be 
scheduled  on  a  regular  basis  employing  training  specialists,  1SD 
personnel,  or  ATD  training  officers  to  provide  individual  and  group 
continuation  training  to  the  instructors.  If  deficiencies  in  an  ATD 
instructor's  proficiency  are  identified  during  routine  observations  of 
his  instructional  skills,  the  instructor  should  receive  remedial  training 
designed  to  remove  these  deficiencies. 
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CHAPTER  VII 


1  THE  INFLUENCE  OF  ATTITUDES  ON  ATD  UTILIZATION 

► 


Many  reviews  of  ATD  training  have  noted  that  the  attitudes  of 
training  personnel  toward  ATDs  exert  a  significant  influence  on  the 
effectiveness  of  their  utilization  (e.g..  Miller,  1953;  Muckier, 
Nygaard,  O'Kelly,  and  Williams,  1959;  Smode,  Hall,  and  Meyer,  1966;  General 
Accounting  Office,  1975;  House  Armed  Services  Committee,  1978).  The 
current  review  is  no  exception.  Units  in  which  ATDs  appeared  to  be  more 
effectively  used  were  characterized  by  a  dominant  positive  attitude 
toward  that  training,  while  negative  attitudes  prevailed  in  units  where 
they  were  less  effectively  used.l 

The  most  direct  evidence  of  the  attitudes  of  training  personnel 
toward  ATDs  is  their  willingness  to  use  the  devices.  Personnel 

responsible  for  the  development  of  aircrew  training  programs  who 
expressed  unfavorable  attitudes  toward  ATD  training  tended  to  prescribe 
use  of  those  devices  for  a  smaller  and  less  significant  portion  of  the 
total  training  program  than  would  be  potentially  possible. 
Additionally,  instructors  and  students  who  held  a  low  opinion  of  the 
training  value  of  the  devices,  when  given  the  option,  tended  to  avoid 
using  ATDs  regardless  of  their  potential  training  value. 

Even  when  ATDs  are  used,  negative  attitudes  toward  the  devices  can 
still  thwart  effective  instruction.  For  example,  a  low  opinion  of  ATD 
training  on  the  part  of  some  managerial  personnel  surveyed  appeared  to 
result  in  the  assignment  of  less  capable  personnel  as  ATD  instructors 
and  less  attention  to  maintenance  support  for  the  devices.  ATD  training 
appeared  to  be  thwarted  also  by  instructors  and  students  who  viewed  ATDs 
negatively.  On  several  occasions,  personnel  who  held  such  views 

reported  practices  that  included  conducting  overly  simplistic  training, 
performing  exotic  maneuvers  that  would  never  be  required  in  the 
aircraft,  devising  ways  to  outwit  the  device  (sometimes  referred  to  as 
"pinballing"  it),  and  drinking  coffee  for  as  much  as  half  of  a  scheduled 
ATD  session. 

Perhaps  the  most  pervasive  and  distressing  consequence  of  negative 
attitudes  toward  ATD  training  was  the  continued  acceptance  in  some  units 
of  recognized  ineffective  and  inefficient  ATD  training  by  managers, 
training  developers,  instructors,  and  students.  These  personnel  were 
usually  able  to  identify  major  deficiencies  in  their  ATD  training 
programs,  but  they  often  seemed  unwilling  to  do  anything  about  the 
problems  because  they  felt  it  would  not  be  worth  the  effort.  As  one 
officer-in-charge  of  ATD  training  put  it,  "Why  worry  when  no  one  cares 
about  ATD  training  anyway?" 


*The  basis  for  judgments  of  the  relative  effectiveness  of  the  ATD 
training  observed  during  this  project  is  described  in  Chapter  II. 


It  is  clear  that  the  valence  of  attitudes  toward  ATD  training  can 
exert  a  significant  influence  on  ATD  utilization.  If  attitudes  are 
positive,  then  everyone  wants  and  tries  to  make  ATD  training  work.  If 
attitudes  are  negative,  then  no  one  cares  if  it  does  not  work.  In  a  few 
cases,  there  even  appeared  to  be  attempts,  perhaps  unconscious,  to  make 
it  not  work  so  there  would  be  no  bother  with  more  ATD  training.  Thus, 
given  the  importance  of  attitudes  toward  ATD  utilization,  the 
development,  conduct,  and  management  of  ATD  training  should  ensure  that 
personnel  actions,  and  the  products  of  those  actions,  promote  the 
formation  of  favorable  attitudes  toward  ATD  training.  To  that  end,  the 
purpose  of  this  section  of  the  report  is  to  provide  information  about 
factors  that  influence  the  formation  of  attitudes  toward  utilization  of 
ATDs . 


FACTORS  INFLUENCING  ATTITUDES  TOWARD  ATD  UTILIZATION 

Very  few  studies  were  identified  during  the  literature  review  that 
dealt  with  attitudes  in  relation  to  the  utilization  of  training  devices. 
The  principal  study  of  direct  relevance,  one  by  Mackie,  Kelley,  Moe,  and 
Mecherikoff  (1972),  examined  factors  that  were  hypothesized  to  influence 
user  attitudes  toward  specific  training  devices  and  developed  a  scaling 
and  profiling  technique  to  assess  those  attitudes.  Since  their  study  is 
the  only  investigation  that  examined  specific  attitudes  toward  ATDs,  it 
was  used  to  a  great  extent  -V:  the  preparation  of  the  following 
discussion.  Readers  who  desire  further  information  regarding  the 

assessment  of  factors  influencing  aft'tudes  towards  ATDs  should  consult 
their  report. 

Other  studies  addressed  the  more  topic  of  the  influence  o 

attitudes  on  the  acceptance  and  utilization  of  new  operational 
equipment,  new  training  devices,  new  or  modified  operational  and 
training  procedures,  advanced  technologies,  and  other  kinds  of  changes 
introduced  in  military,  educational,  industrial  ,  and  other  organization; 
(Lyons,  1966;  McClelland,  1968;  Mecherikoff  and  Mackie,  1970).  Of 
particular  interest  among  these  was  the  study  by  Mecherikoff  and  Mackie 
which  reviewed  the  basic  psychological  literature  on  attitude  change. 
Their  purpose  was  to  provide  practical  guidance  that  could  be  used  to 
promote  the  formation  of  positive  attitudes  toward  organizational 
changes  of  all  kinds.  Readers  interested  in  the  broader  issues  involved 
in  influencing  attitudes  toward  changes  are  referred  to  their  report. 

Factors  that  influence  attitudes  toward  ATDs  can  be  divided  into 

three  categories  to  facilitate  th’s  discussion.  These  categories  a-e: 
(1)  the  design  of  the  ATD  itself;  (2)  the  introduction  of  the  dev  ce  to 

the  using  organization ;  and  (3)  the  manner  in  which  ATD  training  is 

conducted  and  managed. 

A~"D  Des  i  gn 

Three  aspects  of  ATD  design  are  of  interest  here:  the  fidelity  of 
the  device,  i.e.,  its  physical  appearance  and  dynamics;  the 
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Instructor/Operator  station,  and  the  extent  of  user  participation  in  the 
process  of  designing  the  device. 

A  TO  Fi del i ty 

Physical  fidelity,  particularly  with  respect  to  flight  dynamics  or 
handling  characteristics,  has  long  been  recognized  as  a  major  determinant 
of  personnel  attitudes  toward  ATDs  and  ATD  training  (Mackie  et  al . ,  1972, 
Miller,  1954).  The  importance  of  physical  fidelity  is  due  in  part  to  the 
dominance  of  the  aircraft  as  a  model  for  ATD  design.  Flexman  (cited  in 
Muckier  et  al  . ,  1959)  noted  that,  “If  the  simulator  looks,  acts,  feels, 
and  sounds  like  the  aircraft,  the  trainee  is  more  likely  to  be  convinced 
that  practice  in  the  device  will  be  beneficial  to  him."  Training 
personnel  reflected  this  view  when  they  observed  that  the  greater 
complexity  of  simulators  with  motion  platforms  and  external  visual 
displays  seemed  to  make  ATD  training  more  "impressive."  These  personnel 
also  expressed  the  view  that  high  physical  fidelity  ATDs  provide  a  degree 
of  realism  necessary  to  stress  the  student  ("provide  a  pucker  factor"), 
and  that  training  in  them  is  less  boring  and  more  fun  for  both  the 
students  and  instructors. 

Not  all  of  the  attitudes  attributable  to  ATD  design  were  based 
strictly  on  physical  fidelity  considerations.  Some  expressions  of 
attitudes  revealed  an  understanding  of  ATD  training  requirements  and  of 
the  deficiencies  or  strengths  of  ATDs  in  satisfying  those  requirements. 
Tnus,  some  negative  attitudes  apparently  attributable  to  low  physica1 
fidelity  were  based  on  perceptions  of  design  deficiencies  resulting  from 
the  mismatch  of  device  design  with  specific  training  requirements. 
However,  given  the  incomplete  definition  of  training  requirements  at  many 
of  the  units  surveyed,  it  was  usually  difficult  to  determine  whether 
attitudes  attributed  to  low  device  fidelity  were  warranted  or 
unwarranted. 

In  recognition  of  the  influence  of  fidelity  on  attitudes,  and  in  the 
absence  of  clear  statements  of  training  objectives  and  the  relationship 
between  those  otjectives  and  ATD  design  requirements,  many  organizations 
have  consciously  tried  to  “buy"  favorable  attitudes  toward  device 
training  through  the  acquisition  of  high  fidelity  ATDs.  This  practice 
characteri zed  the  procurement  of  simulators  by  several  of  the  major 
airlines  during  the  early  to  mid  1960s  and  has  been  a  factor  ’ n  several 
military  device  procurements  since  that  time. 

On  the  other  hand,  it  would  be  incorrect  to  assume  that  low  fidelity 
devices  always  engender  poor  attitudes.  Such  is  not  the  case.  Favorable 
attitudes  have  been  observed  toward  training  in  quite  low  physical 
fidelity  devices.  For  favorable  attitudes  to  be  found,  however,  it  is 
essential  that  the  lower  fidelity  devices  be  properly  used.  For  example, 
these  devices  must  be  used  only  to  meet  training  objectives  that  are 
ccns;stent  with  their  part-task  training  potential.  Further,  the 

trailing  programs  developed  for  lower  fidelity  devices  must  employ 
cognitive  processes,  as  explained  in  Chapters  III  ana  IV,  to  develop  the 
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cue  discriminations  essential  for  subsequent  transfer  to  operational 
aircraft. 

Although  physical  fidelity  is  a  significant  influence  on  attitudes 
toward  the  utilization  of  ATDs,  it  should  not  dominate  ATD  procurement 
decisions.  Nor  should  it  be  the  sole  basis  of  allocation  of  training 
objectives  to  ATDs.  As  discussed  in  Chapter  V,  the  decision  to  use  ATDs 
as  a  training  resource  should  be  based  on  a  number  of  factors  in 
addition  to  fidelity.  ATD  training  should  incorporate  the  value  of  and 
correct  methods  for  use  of  ATDs  of  all  levels  of  fidelity.  Training 
effectiveness  should  then  be  demonstrated  so  that  training  personnel 
can  have  a  basis  other  than  fidelity  for  forming  favorable  attitudes 
toward  ATDs . 

IQS  Design 


The  Instructor/Operator  Station  is  the  interface  between  the  ATD 

instructor  and  the  device.  The  efficiency  of  the  instructional  process 
is  dependent  to  a  large  extent  upon  how  suitably  designed  this  interface 
is  with  respect  to  the  instructional  process.  Several  of  the  more 
complex  ATDs  surveyed  were  viewed  by  the  assigned  instructors  as  being 
poorly  designed.  They  spoke  disdainfully  of  these  devices'  IOSs  and  the 
instructional  features  operable  from  them.  Specifically,  comments  were 
made  to  the  effect  that  some  necessary  functions  were  made  difficult 
because  of  poor  IOS  designs. 

Problems  that  were  cited  included  inadequate  human  engineering  of 
displays  and  controls  (e.g.,  difficult  to  see  or  reach,  lack  of  labels), 
cumbersome  input  tasks  for  interacting  with  the  computer  (e.g.,  lengthy 
keyboard  entries  for  commonly  used  functions),  cockpit  repeater 
instruments  that  were  arranged  differently  from  those  in  the  aircraft 
(thus  presumably  conflicting  with  highly  developed  scan  patterns), 
instructional  features  that  were  inappropr iately  designed  with  respect 
to  training  requirements  (e.g.,  record/playback  feature  controls  did  not 
allow  instructors  to  reconstruct  needed  portions  of  tactical 
intercepts),  and  poorly  organized  CRT  displays.  Similar  problems  have 
been  noted  in  studies  of  IOS  and  instructional  feature  designs  by 
Charles,  Willard,  and  Healey  (1976),  and  Isley  and  Miller  (1976). 

Mackie  et  al .  (1972),  reported  that  such  IOS  design  deficiencies 
created  problem  set-up  and  performance  measurement  difficulties  that 
adversely  affect  the  acceptability  of  devices  so  far  as  instructors  are 
concerned.  Some  instructors  interviewed  during  the  current  project  did 
reflect  generally  unfavorable  attitudes  that  appeared  associated  with 
their  frustration  in  using  IOSs  and  their  poorly  designed  features. 

The  instructors'  inability  to  use  some  of  the  devices  in  question 
in  an  efficient  manner  could  be  attributed  to  inadequate  instructor 
training.  Therefore,  all  observed  negative  attitudes  toward  IOSs  could 
not  be  attributed  solely  to  poor  design.  Nevertheless,  some  of  the 
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deficiencies  cited  by  the  instructors  were  readily  apparent  even  to  a 
casual  observer  and  could  easily  account  for  the  attitudes  noted. 

The  ATP  Design  Process 


Studies  of  the  use  of  new  equipment  and  other  innovations 
(Mecherikoff  and  Mackie,  1970)  indicate  that  user  attitudes  toward  new 
equipment  are  influenced  by  perceptions  of  the  adequacy  with  which  the 
design  process  took  user  needs  into  consideration.  If  esteemed 
representati ves  of  the  users  were  employed  as  part  of  the  design  team, 
and  if  extensive  inputs  were  seriously  sought  from  the  user  community, 
then  attitudes  toward  the  equipment  tended  to  be  favorable.  This 

relationship  between  the  design  process  and  ATD  user  attitudes  was  noted 
in  interviews  conducted  during  the  current  survey.  Attitudes  of  some 
training  personnel  toward  ATD  training  appeared  to  have  been  shaped  by 
their  perceptions  of  the  adequacy  of  the  procedures  employed  for  the 

design  of  the  ATD  and  its  associated  training  programs.  When  users  were 
more  confident  that  the  designers  of  the  device  and  training  programs 

were  competent  and  had  taken  the  users'  viewpoints  into  consideration, 
then  the  users  felt  more  positive  toward  the  device,  even  though  user 
expectations  may  not  have  been  fully  realized. 

Currently,  simulator  design  and  procurement  agencies  stress  the 

value  of  user  participation  and  input  during  ATD  design.  In  units  that 
had  just  received  new  devices  or  were  expecting  devices  in  the  near 
future,  personnel  often  voiced  confidence  in  the  device  because  personnel 
from  their  unit  had  "been  there"  during  the  design.  In  one  organization 
several  years  after  the  introduction  of  a  device,  it  was  still  well  known 
that  the  design  of  the  ATD  and  training  program  represented  the  input  of 
some  of  that  unit's  best  pilots,  many  of  whom  were  known  by  name  by  most 
of  the  training  personnel.  This  was  cited  as  a  factor  in  building  their 
confidence  in  ATD  training. 

In  contrast,  some  ATDs  were  reportedly  designed  in  isolation  from 
users  and  "dumped  over  the  fence  at  night."  In  such  cases,  the  current 
users  felt  that  they  had  been  isolated  from  ATD  design  and  training 
development.  Interviewees  in  these  units  referred  to  an  anonymous  "they" 
who  "didn't  know  what  we  needed"  in  the  ATD  or  who  "didn't  know  how  we 
train  our  aircrews." 

It  was  not  possible  during  the  current  survey  to  verify  that  these 
devices'  designs  were  really  inappropriate  to  the  training  units’  needs. 
In  several  instances,  the  complaints  appeared  valid.  In  any  event,  such 
comments  reflect  a  "not  invented  here"  syndrome  that  has  been  identified 
elsewhere  as  a  cause  of  negative  attitudes  (Mecherikoff  and  Mackie, 
1970).  It  would  appear  that  developing  devices  in  isolation  from  future 
device  users  is  a  practice  that  could  and  should  be  avoided,  in  the 
interest  of  obtaining  better  ATDs  as  well  as  more  positive  attitudes 
toward  these  devices.  Guidance  concerning  the  employment  of  users, 
training  specialists,  and  other  personnel  on  ATD  design  teams  can  be 
found  in  reports  by  Swain  (1954)  and  Smode  (1972). 


Involving  the  user  in  the  ATD  design  process,  while  highly 
desirable,  cannot  assure  that  the  device  will  conform  to  all  user 
expectations.  Some  disappointments  can  be  expected  simply  because  the 
ATD  procurement  process  must  be  responsive  to  factors  other  than  user 
desires.  These  factors  include  engineering  constraints  and  cost.  In 
addition,  the  time  between  initiation  of  a  requirement  for  an  ATD  and  the 
delivery  of  that  device  (five  years  is  not  uncommon  for  the  more 
sophisticated  devices)  introduces  other  factors.  For  example,  training 
requirements  themselves  can  change  in  such  a  manner  that  the  device’s 
design  is  no  longer  optimum.  Further,  it  is  often  impossible  to  maintain 
continuity  of  personnel  involved  in  an  ATD  development  project  over 
extended  time  periods,  and  the  reasons  for  particular  device  features  may 
be  lost  as  personnel  changes  occur. 

An  effort  was  made  during  the  study  to  determine  the  role  of  ISD 
team  personnel  in  ATD  design.  One  might  assume  that,  by  virtue  of  their 
efforts  to  define  training  requi rements ,  these  personnel  would  be 
influential  in  specifying  characteri sti cs  of  ATDs  for  "ISDed11  training 
programs.  Such  was  not  found  to  be  the  case,  however.  There  appeared  to 
be  three  reasons:  (1)  ISD  teams  are  typically  understaffed  and  do  not 
have  the  resources  to  devote  to  equipment  design;  (2)  most  ISD  design 
efforts  are  conducted  too  late  in  the  device  development  process  for  the 
ISD  results  to  define  device  requirements  (i.e.,  device  designs  are  often 
frozen  before  ISD  teams  can  provide  relevant  information);  and  (3)  ISD 
analytic  techniques  tend  to  concentrate  on  definition  of  criteria,  for 

training  rather  than  on  training  process  information  of  the  kind  that 
would  define  ATD  requirements  unique  to  a  particular  training  program. 
Thus,  rather  than  being  influential  in  defining  ATD  designs,  ISD  teams 
generally  must  accept  as  resource  constraints  the  devices  to  be  made 
available  for  the  training  programs  they  design. 

ATD  Introduction 

Procedures  employed  during  the  introductory  phase  of  ATD  training 
appeared  to  have  an  influence  on  initial  attitudes  toward  that  training. 
These  im'tial  attitudes  also  appeared  to  contribute  to  the  determination 
of  how  the  ATD  was  used  during  the  remainder  of  its  life  cycle.  Of 

particular  note  were  factors  concerned  with  (1)  the  roles  of  managerial 
and  instructional  personnel;  (2)  activities  conducted  in  preparation  for 
the  introduction  of  the  ATD;  and  (3)  the  manner  in  which  the  devices  were 
i ntroduced. 

Personnel  Roles  During  ATD  Introduction 

In  some  units,  the  personnel  who  were  to  be  in  the  initial  ATD 

instructor  cadre  participated  in  selected  device  development  and  testing 
activities  (e.g.,  in-process  reviews,  factory  acceptance  tests,  and 
OT&E).  Several  instructors  who  had  done  so  felt  that  their  initial 
"hands-on“  experience  and  personal  involvement  with  the  development  and 
testing  of  the  ATD  was  a  major  factor  in  the  formation  of  their  own 


Management  of  the  Introduction 


First  impressions  are  important  to  attitude  development  in  almost 
all  endeavors.  First  impressions  toward  ATDs  invariably  influence 

attitudes  toward  ATOs.  To  promote  effective  ATD  training  during  the 
crucial  introductory  phase,  some  units  made  sure  that  all  elements  of  the 
training  system  were  functioning  according  to  expectations  (e.g.,  the 

ATD,  the  training  program,  and  the  instructors)  before  aircrew  training 
activities  were  permitted  to  begin.  In  these  units,  the  devices  were 
perceived  as  effective,  and  attitudes  were  positive. 

In  other  units,  near  chaos  seemingly  prevailed  at  the  time  the 

devices  were  made  available  for  training.  The  ATDs  were  used  for 
training  even  though  they  were  not  really  ready  (e.g.,  they  needed 

additional  "tweaking"  to  prevent  frequent  interruptions),  the  training 

program  was  only  partially  prepared,  and  the  instructors  had  not  been 
fully  trained.  In  the  latter  units,  the  devices  were  viewed  less 

favorably  even  after  the  initial  start-up  problems  had  been  solved. 

The  Conduct  and  Management  of  Instruction  in  the  ATD 

The  Influence  of  Instructors'  Attitudes 


Students,  particularly  those  undergoing  UPT  and  initial  transition 
training,  will  feel  toward  ATDs  as  others  do,  and  behave  during  training 
sessions  according  to  how  others  act.  It  is  not  sufficient  to  attempt 
through  fiat  to  shape  students'  attitudes.  Students  will  model  others; 
they  will  do  what  others  do,  not  what  others  say.  If  an  instructor 
asserts  that  an  upcoming  ATD  session  is  important,  but  shows  by  other 
comments  or  by  example  during  the  session  that  what  occurs  is  not  really 
important,  the  student  will  base  his  judgment  on  the  latter  indications. 
Put  bluntly,  if  instructional  personnel  do  not  value  ATD  training, 
students  will  not  either,  regardless  of  what  is  said. 

During  training  site  visits,  a  problem  regarding  student  "cultures" 
was  evident  at  those  sites  where  instructors  held  ATDs  in  low  esteem. 
Specifically,  students  as  a  group  conformed  to  the  instructors'  views  of 
the  devices.  Even  training  sessions  with  ostensibly  valuable  objectives 
were  often  referred  to  as  "squares  to  be  filled."  Instructors'  attitudes 
toward  ATDs,  whether  individual  devices  or  in  general,  appeared  to  be  the 
most  important  single  variable  in  determining  student  perceptions  of  ATD 
training  value  at  these  sites. 

In  many  units,  a  single  individual  or  a  small  group  of  individuals 
served  as  advocates  for  ATD  training.  These  individuals  were  typically 
highly  motivated,  aggressive  instructors,  training  developers,  civilian 
training  specialists,  or  training  managers  who  took  on  the 
responsibility,  often  without  visible  command  support,  for  assuring  that 
ATD  training  was  conducted  effectively.  Simply  put,  they  cared  about  ATD 
training,  and  the  dedication  they  showed  appeared  to  influence  the 
attitudes  of  other  personnel.  By  contrast,  in  some  units  without  a 
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strong  advocate,  ATD  training  was  recognised  as  being  ineffective,  but  no 
one  seemed  interested  in  doing  anything  about  it.  Everyone  more  or  less 
put  in  the  required  time  in  the  device  and  assumed  the  system  would 
somehow  work,  but  there  was  a  notable  absence  of  positive  views  of  the 
worth  of  ATD  training.  Advocates  are  needed  to  question  such  complacency 
and  to  promote  effective  ATD  training. 1 

Course  Syllabi  and  Training  Scenarios 

Mackie  et  al .  (  1972)  found  that  a  major  factor  related  to  attitudes 
toward  ATD  utilization  that  was  associated  with  the  instructional  process 
was  the  presence  or  absence  of  adequate  training  program  syllabi, 
scenarios,  and  other  training  courseware.  During  the  present  study, 
several  instances  were  noted  in  which  ATD  courseware  was  incomplete  or 
poorly  prepared.  Although  the  possible  influence  of  this  factor  on  the 
attitudes  of  the  personnel  interviewed  was  explored,  none  was  found, 
except  to  the  extent  described  in  the  immediately  following  paragraph. 
(However,  one  might  assume  that  a  failure  to  prepare  adequate  courseware, 
a  situation  noted  at  a  few  units,  is  indicative  of  an  attitude  of 
compl acency .) 

The  above  comments  notwithstanding,  attitudes  were  found  to  be 
influenced  by  the  perceived  realism  of  training  syllabi  and  scenarios  and 
the  completeness  of  performance  evaluation  scenarios  conducted  in  the 
device.  Some  aircrews  reported  that  they  dreaded  ATD  training  because 
that  training  was  made  so  much  more  difficult  than  training  in  the 
aircraft.  Their  ATD  instructors  often  inserted  training  events  such  as 
systems  failure  at  a  pace  that  was  unrealistic  and  too  rapid  for  the 
students'  skill  levels.  Even  with  highly  skilled  pilots,  system  failures 
were  being  inserted  in  such  rapid  succession  that  the  pilots  could  no 
longer  cope  with  the  situation,  let  alone  accomplish  a  useful  training 
mission.  The  general  problem  was  that  a  few  instructors  used  the 
device's  instructional  features  to  make  ATD  training  sessions  extremely 
intense,  even  to  the  point  of  apparently  attempting  to  embarrass  the 
student.  Students,  although  they  often  realized  the  value  of  properly 
conducted  ATD  training,  soon  developed  apathetic  and  even  hostile 
attitudes  toward  ATDs  because  they  were  frustrated  by  their  failures. 
Experienced  aircrews  resisted  the  use  of  these  ATDs  even  for  proficiency 
checks,  because  they  feared  the  tests  would  be  unrealistic  and  it  would 
be  impossible  to  perform  well  on  them. 

In  general,  however,  students  and  instructors  interviewed  during  the 
present  study  did  not  identify  ATD  instructional  techniques  and  methods 
per  se  as  being  a  major  factor  influencing  their  attitudes  toward  ATD 
training.  Students  frequently  mentioned  problems  with  specific 


*A  common  finding  in  the  literature  on  attitude  change  in  such 
situations  is  the  need  for  advocates.  Such  personnel  should  be 
especially  selected,  trained,  and  managed  to  promote  positive  attitudes 
toward  the  subject  of  their  advocacy 
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regarding  key  indicators  of  student  progress  can  be  especially  valuable, 
for  they  permit  the  student  to  observe  systematic  changes  in  task 
proficiency  over  time  when  those  changes  may  not  be  discernible  on  a 
given  day.  (The  role  of  feedback  in  ATD  training  is  discussed  further  in 
Chapter  IV.) 

Instructor  approval .  The  student  must  perceive  that  he  and  his 
efforts  are  respected  and  valued  by  significant  persons,  especially  his 
instructors.  The  nature  and  manner  of  feedback  from  instructors  during 
day-to-day  associations  are  crucial  to  maintaining  positive  student 
attitudes.  Being  told  "You  did  a  good  job"  taps  the  basis  for  social 
behavior  in  general.  Its  power  is  such  that  additional  specific  negative 
feedback  can  be  perceived  as  attempts  to  help  the  learner  rather  than  to 
punish  him. 

Critical  evaluations  of  students  must  necessarily  occur  during 
training.  Unless  the  student  can  recognize  that  he  has  made  progress, 
however,  and  that  the  instructor  values  his  progress,  critiques  of 
performance  will  almost  surely  be  discouraging.  With  recognition  that  he 
is  making  valuable  progress,  negative  feedback  is  much  more  likely  to  be 
interpreted  for  what  it  should  be:  information  necessary  for  further 
improvement. 

In  other,  more  subtle  ways,  students  receive  communications  from 
training  personnel  that  are  very  revealing  of  the  value  placed  on  them. 
The  nature  and  amount  of  nontraining  duties  they  are  assigned,  whether 
training  is  often  conducted  after  normal  duty  hours,  and  various  other 
training  policies  and  practices,  all  tell  the  student  something  about  how 
those  responsible  for  training  view  him,  and  the  priority  placed  on 
training.  Is  learning  to  be  a  quality  aircrewman  the  overriding  goal? 
Often,  answers  are  revealed  through  policies  that  facilitate  this  goal, 
or  contrarily,  interfere  with  its  attainment. 

Management  of  ATD  Training  Resources 


Management's  job  is  to  enable  effective  ATD  training  to  take  place. 
When  management  does  not  perform  this  job  in  a  manner  satisfactory  to  the 
personnel  involved,  ATD  training  may  become  a  focus  for  criticism. 
Managerial  practices  relating  to  training,  resource  scheduling,  and 
maintenance  were  commonly  criticized  as  unpopular  during  the  present 
survey,  and  similar  findings  have  been  reported  previously  (Mackie  et 
al.,  1972). 

Some  units  scheduled  ATD  training  late  at  night  and  still  expected  a 
full  workday  the  next  day  for  the  students  involved.  Additionally,  late 
or  early  ATD  sessions  often  resulted  in  extremely  long  hours  for 
instructors  without  adequate  recognition  of  their  efforts  or  compensating 
time  off.  In  some  units,  students  were  protected  by  regulations  from 
such  abuse,  but  not  in  any  of  the  continuation  training  units  visited. 
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The  scheduling  of  ATD  training  during  normally  off-duty  hours  is  a 
wide-spread  practice.  The  reliability  of  modern  devices  can  support  20 
or  more  hours'  training  per  day.  In  some  units,  the  management  policy 
was  to  designate  personnel  for  specific  shifts  of  not  more  than  8  hours 
per  day  to  make  full  use  of  the  available  time,  a  practice  employed  also 
by  many  commercial  airlines.  In  several  continuation  training  units 
visited,  however,  some  instructors  were  scheduled  for  in  excess  of  8 
hours  per  day  with  a  consequent  unfavorable  impact  on  their  views  toward 
ATDs.  This  practice  was  incorrectly  compared  by  unit  management  with  the 
airlines'  use  of  devices  for  almost  continuously  scheduled  training 
activities.  Airlines  schedule  almost  continued  device  use,  but  personnel 
work  normal  hours.  This  mistaken  concept  of  optimum  use  of  ATDs  had  an 
obvious  impact  on  instructor  attitudes. 

Requirements  that  mandated  a  minimal  number  of  hours  of  use  of  an 
ATD  were  also  extremely  unpopular.  (Mandated  ATD  utilization  rates  are 
discussed  in  Chapter  II.)  Students  and  instructors  told  of  wasting  time 
in  the  ATD,  and  even  of  being  dishonest  in  the  recording  of  ATD 
utilization  time,  to  help  their  unit  meet  mandated  rates.  The  effect 
that  this  practice  had  on  their  attitudes  toward  ATD  training  was  to 
suggest  that  the  training  itself  was  unimportant  and  to  be  avoided  when 
possible,  an  attitude  that  was  encountered  repeatedly  during  the  present 
project. 

Lack  of  device  availability,  disruptions  during  training,  and 
training  i  wi th  degraded  systems  because  of  inadequate  ATD  maintenance 
support  and  parts  availability  were  cited  as  factors  leading  to  negative 
attitudes  toward  ATDs.  Such  problems  were  noted  primarily  among  older 
devices.  The  reliability  of  current  generation  devices  is  such  that 
complaints  concerning  their  availability  for  scheduled  training 
activities  were  rarely  noted,  and  those  complaints  that  were  noted  were 
restricted  primarily  to  visual  system  alignment/adjustment  problems  and 
failures  of  flight  director  instruments  in  the  cockpit.  However,  over  an 
extended  period  following  introduction  of  a  new  ATD,  training  in  one  unit 
reportedly  was  interrupted  on  the  average  of  two  to  three  times  per 
session  by  computer  malfunctions.  While  these  interruptions  were  short, 
the  loss  of  continuity  of  training  was  frustrating.  Instructors  reported 
becoming  so  frustrated  with  such  interruptions  that  they  would  cancel  a 
training  session.  Having  to  use  ATDs  with  such  problems  resulted  in 
lowered  opinions  of  new  ATDs  among  personnel  who  must  use  these  devices. 
Fortunately,  such  disruptions  are  uncommon  after  the  devices  have  been  in 
use  for  several  months  ancl  "debugging"  has  been  completed. 

$ 

In  an  effort  to  reir'force  positive  attitudes  toward  ATD  training, 
some  users  provide  relatively  plush  facilities  for  that  training.  In  one 
unit  where  this  practice  was  employed,  personnel  cited  the  design  of  the 
facilities  as  a  positive  attitudinal  factor.  However,  in  units  that 
could  be  characterized  as  having  ineffective  ATD  training,  comparably  new 
facilities  were  described  as  "sterile  showcases,"  and  attitudes  toward 
ATD  training  were  generally  negative. 
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Frustrations  generated  by  labored  use  of  poorly  designed  displays  and 
controls  will  result  in  lowered  opinions  of  ATD  training  in  general. 

t  Training  personnel  who  will  be  eventual  users  of  an  ATD  should 
participate  in  the  design  of~the  device!  It  isTnot^just  the  design ~~of 
the~3ev ice  that  influences  user  attitudes,  but  also  their  perception  of 
how  well  the  design  process  took  their  expressed  needs  into 
consideration.  Participation  of  users  in  the  ATD  design  process  will 
influence  the  formation  of  positive  attitudes  toward  the  ATD. 

•  Training  personnel  who  will  be  eventual  users  of  an  ATD  ^houVd 
participate  in  the  fnitiaT~evaluation  of  the  device.  User  participation 
Tn  device  testing  will  tend  to  influence  their  opinions  positively  toward 
ATD  training. 

•  Individuals  within  the  training  organization  should  be  encouraged 
to  become  advocates  for  ATD  training.  Advocates  are  needed  to  guard 
against  the  complacent  acceptance  of  ineffective  ATD  training  and  for 
adequate  support  for  that  training.  Such  personnel  should  be  specially 
selected,  trained,  and  managed  to  promote  positive  attitudes  toward  ATD 
training. 

•  Planning  for  the  introduction  of  a  new  ATD  and  tor  the 
implementation  of  ATD  training  should  begin  early  in  the  ATD  procurement 
process  and  should  be  given  strong  management  support.  Transitions  to  a 
new  device  and  a-  new  training  program  wiTl  be  smoother  if  difficulties 
associated  with  change  are  anticipated  and  provisions  are  made  to 
minimize  problems. 

•  Changes  in  training  that  might  disrupt  established  procedures 
should  be  made  independently  of  "ATD  introduction.  Such  changes  should  be 
initiated  before  a  new  ATD  is  introduced  so  that  the  instructional  staff 
can  become  familiar  with  them  and  can  develop  confidence  in  their  use. 

•  All  elements  of  the  training  system  should  be  functioning 
according  to  expectations  before  ATD  training  is  permitted"  to  begun  THe 
perceived  effectiveness  of  ATD  training  during  ~Tts  introduction  is 
crucial  to  the  formation  of  initial  attitudes.  The  ATD  should  not  be 
used  until  it  meets  all  specifications.  Support  for  training  (e.g., 
instructor  training,  maintenance  support)  should  also  be  fully 
implemented  before  training  commences. 

•  ATD  Instructors  should  serve  as  effective  role  models  who 
encourage  the  formation  of  positive  student  attitudes  toward- "ATP 
training!  During  the  operational  utilization  of  the  ATD,  the  key  factor 
i nfl uencing  student  attitudes  is  the  attitudes  of  their  instructors. 
Procedures  used  to  select,  train,  and  manage  the  instructors  should  be 
designed  to  foster  proper  instructor  attitudes  as  well  as  to  develop 
effective  instructional  skills. 
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•  The  content  and  pace  of  ATP  training  should  be  appropriate  to  the 
student' s  skill  1 evel .  Overloading  the  student  with  inappropriately 
difficult  training,  in  addition  to  being  inefficient  instruction,  fosters 
the  development  of  unfavorable  student  attitudes  toward  ATD  training. 

•  Student  learning  should  be  effectively  managed  so  that  they 
maintain  favorable  attitudes  toward  ATD  training.  Four  requirements 
should  be  met.  Students  shoul d:  (l)  understand  the  significance  of  each 
task  and  the  need  for  its  mastery;  (2)  make  progress  toward  mastery  of 
relevant  skills  during  each  practice  session;  (3)  perceive  that  progress 
is  being  made;  and  (4)  perceive  that  they  and  their  efforts  are  respected 
and  valued  by  significant  persons,  especially  instructors. 

•  Excessive  demands  should  not  be  made  on  the  time  of  ATD 
instructional  personnel.  Schedules  of  ATD  utilization  that  create 
excessive  demands  on  instructor's  time,  particularly  during  normally  off- 
duty  hours,  will  adversely  affect  attitudes. 

•  Professionalism  in  ATD  training  should  be  deliberately  fostered. 
The  fostering  of  professionalism  in  ATD  training  is  the  nost  important 
responsibility  that  ATD  training  managers  can  assume  in  ensuring 
favorable  attitudes  toward  ATD  training.  A  professional  atmosphere 
should  be  established  in  which  effective  instruction  is  given  the  highest 
unit  priority. 
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assessed  without  endangering  the  pilot  or  the  aircraft.  Similarly,  the 
effectiveness  of  weapons  delivery  training  in  a  weapons  system  trainer 
can  be  evaluated  without  the  expenses  and  potential  risks  associated  with 
weapons  release  in  actual  flight  by  determining  whether  skills  develop  in 
the  device.  Again,  however,  it  is  necessary  that  the  equivalence  of  ATD 
and  aircraft  performance  be  established  empirically  or  analytically. 

When  circumstances  and  resources  permit,  however,  assessment  of 
performance  in  flight  through  a  transfer  of  training  study  is  a  more 
satisfactory  approach  to  determining  ATD  training  effectiveness.  The 
reason  for  this  is  obvious:  Performance  in  the  aircraft,  not  in  the 
simulator,  is  the  ultimate  criterion  of  operational  readiness.  There  is 
always  a  risk  that  the  objectives  of  training  in  a  simulator  are  not 
fully  consistent  with  operational  flight  requirements,  so  performance  in 
the  aircraft  should  be  examined  as  a  means  of  validating  ATD  training. 
Without  having  validated  ATD  training  in  this  manner,  it  is  impossible  to 
assess  th  1  magnitude  of  that  risk. 


A  CAUTION 

The  evaluations  of  ATD  training  effectiveness  performed  by  the  units 
visited  during  this  study  were  largely  intuitive  and  were  conducted  by 
personnel  with  no  formal  qualifications  for  the  conduct  of  such 
evaluations.  Almost  no  provisions  were  made,  especially  after  the 
original  introduction  of  a  device,  for  the  systematic  collection  of 
performance  data  that  could  lead  to  unequivocal  judgments  regarding 
training  effectiveness.  The  major  reason  for  this  lack  was  the  fact  that 
personnel  available  to  conduct  evaluations  seldom  had  any  background  in 
the  technical  aspects  of  evaluative  research  required  for  valid 
assessments  of  ATD  training  programs. 

Since  local  unit  training  personnel  are  often  expected  to  evaluate 
ATD  training  effectiveness,  it  is  necessary  that  these  personnel  and 
their  commanders  be  aware  when  gathering  and  interpreting  data  of 
difficulties  that  can  arise  from  reliance  on  relatively  informal  and 
loosely  controlled  studies.  Following  the  guidance  provided  in  this 
chapter  will  benefit  the  quality  of  ATD  training  evaluations  conducted  by 
unit  personnel.  This  guidance  calls  attention  to  many  common  pitfalls  in 
data  collection  and  interpretation  which  should  be  avoided  in  ATD 
training  evaluation. 

This  guidance  can  be  helpful  to  those  responsible  for  evaluating  ATD 
training  or  for  the  review  of  reports  of  such  evaluations.  However, 
within  the  context  of  this  report,  it  is  not  possible  to  provide  a 
"cookbook"  for  the  design  and  conduct  of  a  fully  scientific  and  rigorous 
evaluation.  For  this  reason,  it  is  recommended  that  technical  assistance 
be  sought,  wherever  possible,  when  a  formal  (or  even  informal)  evaluation 
is  in  order.  The  major  source  of  Air  Force  technical  expertise  in  ATD 
training  evaluation  is  the  Air  Force  Human  Resources  Laboratory.  Other 


sources  within  the  Air  Force  and  elsewhere  can  be  identified  through  the 
Laboratory. 


PRELIMINARY  CONSIDERATIONS 

It  sometimes  is  assumed  that  the  training  value  of  an  ATD  is 
inherent  in  the  device,  i.e.,  that  its  training  effectiveness  can  be 
determined  independently  of  other  considerations.  This  assumption  is  not 
valid.  Regardless  of  its  design,  an  ATD  must  be  used  to  be  effective, 
and  it  is  the  manner  of  its  use  which  largely  determines  its 
effectiveness.  Thus,  any  investigation  of  ATD  effectiveness  is,  in  fact, 
an  investigation  of  the  device's  effectiveness  when  used  in  a  specific 
way.  If  the  manner  of  its  use  changes,  its  effectiveness  also  may 
change.  While  the  engineering  design  may  limit  the  tasks  which  can  be 
learned  in  an  ATD  and,  hence,  its  potential  training  effectiveness,  the 
way  the  device  is  used  determines  whether  that  potential  will  be 
real ized. 

Further,  ATD  training  effectiveness  is  related  to  the  objectives  of 
that  training.  Effectiveness  cannot  be  determined  except  with  respect  to 
the  extent  to  which  training  achieves,  or  facilitates  the  achievement  of, 
specified  performance  goals.  A  given  ATD  training  program  can  be  quite 
effective  with  respect  to  certain  training  objectives  and  ineffective— or 
possibly  even  have  a  negative  training  value— with  respect  to  others. 
Before  the  value  of  any  ATD  training  program  can  be  determined,  the 
intent  or  objectives  of  that  training  must  be  defined.  Effectiveness, 
then,  can  only  be  determined  with  respect  to  the  extent  to  which  defined 
training  objectives  are  met. 

For  these  reasons,  preparation  for  the  conduct  of  an  evaluation 
effort  is  an  important— often  the  major — part  of  that  effort.  If  the 
purpose  of  the  assessment  is  to  determine  whether  training  will  be  more 
effective  with  the  introduction  of  a  new  device,  care  must  be  taken  to 
assure  that  the  new  device  will  be  used  in  optimum  ways.  (Information  in 
Chapters  III  through  VII  should  be  carefully  considered  when  this  is 
being  dGne.)  If  the  purpose  of  the  assessment  is  to  determine  whether 
training  with  a  device  is  preferred  to  training  without  it,  it  is 
important  that  both  training  programs  be  optimized  with  respect  to  all 
training  resources  they  employ.  Otherwise,  the  study  results  may  reflect 
the  difference  between  well  and  poorly  conceived  and  conducted  training 
programs,  not  the  effectiveness  of  device  training  per  se. 


DESIGN  OF  ATD  TRAINING  EFFECTIVENESS  STUDIES 

A  study  to  determine  the  effectiveness  of  a  particular  ATD  training 
activity  must  be  organized  in  a  manner  that  will  permit  valid  and 
repeatable  results  to  be  obtained.  Usually,  a  format,  or  evaluation 
study  design,  is  adopted  that  will  permit  the  performance  of  ATD-trained 
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particular  caution  should  be  exercised  in  comparing  performance  under  the 
new  program  with  existing  data  from  the  old  program.  For  such  a 
comparison  to  be  valid,  the  pre-existing  data  must  have  been  gathered 
under  conditions  which  would  have  been  applicable  to  a  control  group 
trained  concurrently  with  the  test  group.  Often,  it  will  not  be  possible 
to  determine  whether  these  conditions  were  the  same,  so  the  results  of 
this  design,  when  applied  in  an  "after  the  fact"  manner,  must  be  viewed 
cautiously.  Particular  attention  should  be  paid  to  whether  the  two 
programs  used  the  same  or  different  instructors  and  whether  student 
characteristics,  grading  practices,  or  performance  criteria  have  changed 
from  the  old  to  the  new  program. 

The  Uncontrolled  Transfer  Design 


There  are  circumstances  in  which  a  control  group  cannot  be  employed 
and  for  which  suitable  control  data  do  not  exist.  Such  circumstances 
might  be  dictated  by  administrative  or  safety  considerations.  For 
example,  it  might  be  unacceptable  to  "penalize"  the  control  group  by 
requiring  that  it  undergo  a  presumably  inferior  training  program,  or  when 
withholding  simulator  training  in  certain  safety  procedures  might  involve 
unacceptable  risks  for  the  students  involved.  In  other  instances,  a 
control  group  simply  may  not  be  feasible;  the  effectiveness  of  lunar 
landing  simulators  could  not  be  determined  by  employing  a  no-simulator 
training  control  group  of  astronauts. 

When  a  control  group  cannot  be  employed  and  suitable  control  data  do 
not  exist,  ATD  training  effectiveness  can  be  established  by  determining 
whether  students  can  perform  a  particular  task  in  the  operational 
aircraft,  following  its  learning  in  the  simulator,  without  an  opportunity 
to  learn  that  task  in  the  aircraft.  Data  gathered  under  this  design  will 
be  suspect  to  the  extent  that  it  cannot  be  shown  conclusively  that  the 
skills  involved  can  be  attributed  to  ATD  training.  Nevertheless,  such 
data  can  carry  considerable  weight,  particularly  when  a  task  critical  to 
pilot  safety  is  involved  and  a  plausible  case  can  be  made  that  the 
underlying  skills  probably  are  attributable,  at  least  in  part,  to  the  ATD 
training  program. 

The  Backward  Transfer  Design 

The  Backward  Transfer  Design,  an  ATD  evaluation  design  that  examines 
pilot  performance  in  the  simulator  instead  of  in  the  aircraft,  is  based 
upon  the  transfer  of  training  concept.  In  a  Backward  Transfer  study,  a 
pilot  who  already  has  demonstrated  mastery  of  relevant  flight  training 
objectives  is  “transferred"  to  the  ATD,  where  he  is  required  to  perform 
tasks  corresponding  to  those  he  has  mastered  in  flight.  If  he  can 
perform  such  tasks  to  criterion  levels  without  practice  in  the  device, 
backward  transfer  is  said  to  have  occurred,  and  this  fact  is  taken  as 
evidence  that  transfer  in  the  device-to-aircraft  sequence,  although  of 
unknown  quantity,  will  be  positive. 


114 


The  backward  transfer  design  should  be  used  with  caution  for  at 
least  three  reasons:  (1)  positive  results  assume  (possibly  incorrectly) 
that  a  suitable  training  program  exists  for  the  ATD;  (2)  experienced 
personnel  already  proficient  at  operational  tasks  may  have  highly 
generalized  skills  not  possessed  by  recent  training  program  graduates  and 
may  be  able  to  transfer  performance  to  the  device  because  of  such  general 
skills  rather  than  skills  needed  to  operate  a  particular  vehicle  or 
perform  a  particular  mission;  and  (3)  the  ATD  may  be  suitably  designed 
for  the  evocation  of  a  particular  set  of  behaviors  by  skilled  performers, 
but  may  lack  the  cues  essential  to  the  development  of  those  behaviors  by 
students. 

While  backward  transfer  data  should  not  be  the  sole  justification 
for  simulator  use  or  procurement,  one  would  be  hesitant  to  recommend  a 
device  which  could  not  be  handled  by  competent  pilots.  Negative  results 
could  be  misleading  also,  however,  at  least  insofar  as  the  value  of  an 
ATD  during  early  stages  of  training  is  concerned.  It  is  possible  for 
some  tasks  to  be  performed  in  an  aircraft  by  personnel  who  cannot  do  so 
in  a  device  because  they  use  cues  not  present  in  the  device.  The  same 
device,  however,  may  provide  other  cues  which  trainees  can  learn  to  use 
to  perform  those  same  tasks  in  the  device  for  subsequent  transfer  to  the 
operational  vehicle.  For  advanced  skills,  even  successful  backward 
transfer  can  be  difficult  to  interpret.  Because  of  their  overall  skill 
level,  experienced  aircrews  nay  adapt  readily  to  peculiarities  of  an  ATD 
when  in  fact  the  device  is  worthless  for  training  new  higher  order  skills 
that  are  unique  to  a  given  target  aircraft. 

The  ATD  Performance  Improvement  Design 

An  essential  feature  of  an  effective  ATD  training  program  is 
improvement  in  the  performance  of  trainees  as  a  result  of  training  they 
receive  in  the  device.  If  such  improvement  does  not  occur  in  the  ATD, 
there  will  be  little  expectation  that  subsequent  operational  performance 
will  be  improved  as  a  result  of  device  training.  Because  of  this 
dependency  relationship,  improvement  in  performance  in  the  ATD  often  is 
cited  as  evidence  that  ATD  training  is  effective.  This  typically  is  done 
when  circumstances  preclude  the  employment  of  a  transfer  design  to 
determine  simulator  training  effectiveness. 

Clearly,  there  are  circumstances  in  which  the  ATD  Performance 
Improvement  Design  can  provide  the  best  estimate  of  whether  ATD  training 
is  effective.  The  use  of  the  approach  exemplified  by  this  design  to 
assess  performance  that  cannot  be  assessed  in  flight  due  to  safety  or 
economic  considerations  was  discussed  in  the  introduction  to  this  section 
of  the  report.  It  must  be  noted,  however,  that  this  design  does  not 
assure  that  skills  acquired  during  ATD  training  will  transfer  to  the 
aircraft.  The  design  can  show  that  a  necessary  condition  for  transfer 
has  been  met,  but  it  does  not  justify  the  conclusion  that  the  improved 
performance  in  the  simulator  will  result  in  improved  operational 
performance.  This  model  possibly  is  most  useful  in  a  negative  way:  If 
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The  basic  requi remen t  or  selecting  a  performance  parameter  to 
measure  is  its  relevance-  tc  Me  objectives  of  simulator  training.  cor 
example,  an  objective  s  ght  oe  landing  ar,  aircraft,  in  which  case  a 
simple  pass/fail  measure  might  be  acceptable,  i.e.,  the  landing  met  pre- 
established  criteria  or  it  tailed  to  do  so.  More  specifM  and  detail er 
objectives  and  mv,r.  j-'os  of  performance  would  likely  be  preferable, 
however.  In  the  case  cr  landing  an  aircraft,  for  example,  separate 
measures  might  be  riesi  red  which  indicate  whether  the  aircraft  control, 
procedural,  navigational,  ana  communication  tasks  performed  by  the  pilot 
during  the  five  minutes  preceding  contact  with  the  runway  were  performed 
within  predetermined  tolerances.  The  specificity  of  the  parameters 
measured  should  be  determined  by  the  specificity  of  the  training 
objectives.  Since  aircrew  training  programs  should  be  based  upon  quite 
specific,  systematically  derived  behavioral  objectives,  it  is  to  be 
expected  that  a  well-conducted  ATP  training  effectiveness  study  will 
yield  multiple,  quite  specific  measures  of  trainee  performance  keyed  to 
previously  described  training  objectives. 
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judgment  of  the  quality  of  the  performance.  Command  or  squadron  rating 
sheets  and  other  evaluative  indices  of  aircrew  performance  generally  are 
not  useful  for  such  purposes.  Ratings  are  by  definition  subjective  and 
usually  are  found  to  incorporate  unintended  rater  bias.  Moreover,  even 
if  administered  in  a  manner  free  of  bias,  they  tend  to  be  overly  general 
and  do  not  provide  the  explicit,  systematic  information  on  student 
performance  needed  for  the  detailed  analysis  of  ATD  training 
effectiveness.  Daily  grade  slips  currently  in  use  by  the  Air  Force, 
Army,  and  Navy  also  are  unsuitable  for  collecting  data  describing  student 
performance,  because  they  are  designed  for  evaluative,  rather  than 
performance  recording,  purposes.  Recognizing  the  inadequacy  of  the  forms 
as  they  exist,  many  instructors  note  instances  of  performance  on  the 
backs  of  grade  slips.  However,  such  notations  usually  are  limited  to 
descriptions  of  errors  and  are  incomplete  and  unsystematic,  so  they  would 
be  inadequate  for  assessing  ATD  effectiveness. 

Standardized  Conditions 

The  data  should  be  recorded  under  standardized  conditions. 
Measuring  instruments  should  be  demonstrably  comparable  in  content  and 
administration  across  both  experimental  and  control  groups.  For  example, 
ai r- to-ground  attack  performance  evaluations  should  follow  the  same  or 
comparable  routes  for  both  groups,  and  all  special  events,  such  as 
nostile  missile  fire  and  equipment  malfunctions,  should  be  evaluated 
under  prescribed  conditions.  Factors  that  cannot  be  standardized,  such 
as  daily  variations  in  environmental  conditions,  should  be  controlled  by 
assuring  that  comparable  numbers  of  subjects  are  evaluated  under  each 
variation. 

Uniform  Criteria 


The  data  should  be  based  upon  uniform  criteria  or  standards  applied 
equally  to  all  experimental  and  control  trainees.  The  specifications  of 
tolerances  around  correct  performances,  e.g.,  airspeed  130  +5  knots  at 
touchdown,  must  not  vary  among  observers  or  trainees.  The  degree  of 
uniformity  required  can  only  be  obtained  through  a  concerted  program  of 
observer  training  and  standardization.  The  assumption  should  never  be 
made  that  uniform  data  will  be  obtained  without  such  training,  even  when 
competent  instructor  or  check  pilots  are  the  observers. 

Independent  Personnel 

The  data  should  be  obtained  by  personnel  who  are  independent  of  the 
ATD  training  activities,  and  they  should  have  no  prior  knowledge  of  an 
individual  trainee's  performance  during  device  training.  Ideally,  they 
should  not  even  know  which  trainees  are  assigned  to  ATD  training  and 
which  are  assigned  to  no-ATD  control  groups.  Since  significant  biases  in 
observer  reports  tend  to  be  pervasive  and  are  very  difficult  to  correct 
during  data  analyses,  observers  should  be  obtained  from  administrative 


units  as  remote  as  is  practical  from  the  unit  responsible  for  conduct  of 
the  study  so  that  biases  will  be  less  likely. 


REPORTING  ATD  TRAINING  EFFECTIVENESS 

Except  in  those  rare  instances  in  which  the  officer  conducting  the 
study  has  sole  responsibility  for  decisions  concerning  the  ATD  training 

activity  under  study,  a  report  documenting  the  study  findings  must  be 

prepared.  Often,  the  format  and  content  of  such  a  report  are  dictated  by 
organizational  requirements.  These  report  requirements  typically  are 

developed  in  connection  with  test  and  evaluation  procedures  for 
operational  equipment  rather  than  for  training  equipment.  Consequently, 
the  resulting  reports  often  tend  to  be  oriented  toward  equipment 

performance  and  the  manner  in  which  it  is  tested  rather  than  toward 
trainee  performance. 

In  addition  to  meeting  report  content  and  format  requirements 
prescribed  by  the  parent  organization,  the  investigator  conducting  a 
simulator  training  effectiveness  study  has  an  implicit  obligation  to 
report  two  kinds  of  information:  (1)  description  of  the  manner  in  which 
the  ATD  training  effectiveness  study  was  conducted;  and  (2)  the  study 
findings.  The  purpose  for  reporting  the  study  findings  is  obvious.  The 
purpose  for  describing  the  study  itself  is  to  permit  persons  not  involved 
in  the  study  to  make  independent  judgments  concerning  the  probable 
validity  of  the  study  findings,  based  upon  how  well  the  study  was 
conceived  and  conducted.  Good  descriptions  of  methodology  can  also 
facilitate  the  conduct  of  future  device  evaluation  efforts  by  providing 
an  example  to  other  investigators. 

Describing  the  Study 

The  description  of  the  study  included  in  the  report  should  include 
the  kinds  of  information  indicated  below.  The  level  of  descriptive 
detail  appropriate  in  a  particular  report  is  a  matter  of  judgment.  That 
judgment  should  be  made  on  the  basis  of  a  criterion  of  clearly 
communicating  to  an  anticipated  reader  what  he  might  wish  to  know  to 
support  his  own  assessment  of  the  study  findings. 

Identification  of  the  ATD  Involved 

Often,  this  is  a  simple  matter  that  can  be  accomplished  in  part 
through  reference  to  design  documents  or  manufacturers’  descriptions.  In 
the  case  of  unique,  locally  fabricated  or  modified  devices,  more 
information  may  be  required. 

Description  of  the  Manner  in  Which  the  ATD  was  Used 

This  description  should  identify  the  principal  features  of  the 
device  training  program;  reference  formal  course  description  materials 
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Deficiencie s  in  Formal  ATP  E  f fee t j_v eness  Studies 

The  most  critical  deficiencies  that  were  identified  in  formal 
studies  of  ATD  effectiveness  aro.,e  from  (1)  the  inability  of  test 
personnel  to  exercise  necessary  experimental  control  during  studies  in 
operational  training  environments;  (2)  inadequate  support  for  ATD 
training  activities;  and  (3)  methodological  problems  that  limited  the 
studies  and  confounded  interpretation  of  their  results.  These 
deficiencies  are  relatively  common  and  are  worth  noting  here. 

Inability  to  Exercise  Co n trol 

ATDs  and  other  training  resources  are  sometimes  made  available  for 
research  or  evaluation  with  the  restriction  that  these  activities  not 
interfere  with  ongoing  operations  and  training.  However,  in  order  to 
carry  out  these  studies,  it  is  usually  necessary  to  specify  the  amount, 
sequence,  and  type  of  training  a  subject  is  to  receive.  Adjustments 
often  must  be  made  in  existing  training  and  operational  schedules  for 
these  requirements  to  be  met.  A  common  problem  reported  during  the 
present  project  was  the  deviation  f'-om  planned  study  schedules  that 
invalidated  the  ATD  comparisons  that  were  to  be  made.  These  deviations 
were  usually  attributed  to  operational  personnel  who  did  not  understand 
or  give  priority  to  the  studies.  Other  related  problems  that  were 
reported  included  biased  assignment  of  subjects  to  comparison  groups, 
unscheduled  changes  in  instrumental  personnel,  and  lack  of  cooperation 
from  management  and  operational  personnel  in  the  collection  of  data. 

Inadequate  Support 


Another  major  source  of  deficiencies  was  the  failure  in  many 
programs  to  provide  adequate  support  for  the  ATDs  during  the  study.  For 
example,  ATDs  were  used  that  were  new  and  not  yet  ready  for  training,  or 
they  were  not  properly  maintained.  Also,  instructors  were  not  trained 
adequately.  The  results  of  ATD  effectiveness  evaluations  conducted  with 
such  degraded  support  are  likely  to  underestimate  the  potential 
effectiveness  of  ATD  training  and  thus  lead  to  under-utilization  of  the 
devices.  Assessments  of  ATD  training  effectiveness  should  o°ly  be 
conducted  after  personnel  responsible  for  the  assessments  are  convinced 
that  the  studies  will  be  adequately  supported. 

Methodological  Deficiencies 

Serious  methodological  flaws  are  common  in  published  studies  of 
training  effectiveness,  many  of  which  deal  with  flight  simulation.  The 
problems  concern  deliberate  design  and  analytic  procedures  that  render 
the  reported  statistical  tests  meaningless.  The  more  common  flaws 
included  matching  test  subjects  to  compensate  for  small  numbers  of 
subjects  when  matching  was  inappropriate  to  the  statistics  employed, 
using  inappropriate  criteria  for  accepting  (or  rejecting)  a  study 
finding,  uncritical  acceptance  of  the  conclusions  of  other  ..searchers 
who  performed  related  studies  without  examining  their  data  and  analytic 
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methods,  incomplete  reporting  of  experimental  procedures,  and  just  plain 
carelessness  in  data  handling,  including  simple  arithmetic  errors. 
Uncritical  acceptance  of  the  reported  findings  of  studies  of  AID 
effectiveness  could  result  in  an  inaccurate  impression  of  the  worth  of 
the  devices  in  question. 

Two  other  methodological  problems  noted  during  the  present  project 
were  also  identified  in  the  previously  cited  review  by  Diehl  and  Ryan 
(1977).  One  problem  is  the  fact  that  student  performance  criteria  used 
to  assess  the  effectiveness  of  ATD  training  are  usually  defined  in 

subjective,  ambiguous,  nonstandard  terms,  often  involving  poorly  defined 
measures.  The  other  problem  is  that  training  requirements  analyses, 
which  are  necessary  (but  not  sufficient)  to  establish  the  validity  of 
training  requirements,  often  are  not  performed  prior  to  conducting  the 

evaluation.  Problems  such  as  these  make  study  results  uninterpretable 
even  when  there  are  no  design  and  analytic  problems  to  contend  with. 

Because  they  are  not  trained  to  assess  the  technical  adequacy  of  the 
methodologies  employed  in  the  design,  conduct,  and  analysis  of  these 

studies,  senior  command  personnel  often  are  not  aware  of  deficiencies  in 
many  of  the  formal  ATD  assessment  studies  that  have  been  conducted  and 
reported.  Furthermore,  inadequacies  in  the  managerial  control  and 
support  provided  to  the  studies,  inadequacies  that  often  eviscerated  even 
the  well-designed  studies,  usually  are  not  reported  or  are  viewed  as 
being  justified  by  operational  necessities,  an  explanation  that  appears 
to  satisfy  many  personnel  but  does  nothing  to  salvage  uninterpretable 

data. 

There  is  likely  to  be  an  increasing  number  of  ATD  training 

assessment  studies  in  the  future.  If  the  methodologies  employed  and  the 

managerial  control  and  support  of  the  assessments  are  not  improved,  then 

these  studies  may  be  not  only  useless,  but  misleading  and  consequently 
damaging  to  future  ATD  utilization. 

Informal  Studies  of  ATD  Training  Effectiveness 

Not  all  ATD  training  effectiveness  assessments  are  conducted  and 
reported  formally.  Even  when  the  results  of  formal  studies  are  not 

available,  most  of  the  personnel  involved  in  ATD  training  could  cite  some 
kind  of  evidence"  that  their  training  was  effective.  For  exampit, 
Operational  Readiness  Inspections  or  other  Inspector  General-type  unit 
evaluations  that  included  analyses  of  unit  ATD  utilization  practices  were 
cited  in  a  few  instances  to  support  existing  ATD  training  programs. 

It  was  found  during  interviews  that  there  have  been  numerous 
informal  and  generally  unsystematic  studies  of  ATD  effectiveness,  usually 
conducted  by  unit  personnel  with  no  formal  training  or  other  indicated 
qualifications  for  undertaking  such  a  task  Many  of  these  studies  have 
been  imaginative  and  resourceful,  but  they  often  suffered  from  various 
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validity  of  the  study  findings.  A  study  report  should  include,  as  a 
minimum,  an  identification  of  the  ATD  involved,  description  of  the  manner 
in  which  it  was  used,  identification  of  the  trainees,  a  description  of 
the  study  design,  a  description  of  the  performance  measurement  and 
analytic  procedures,  and  a  detailed  description  of  the  study  results. 
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CHAPTER  IX 


MAINTAINING  ATD  TRAINING  EFFECTIVENESS 


Chapters  III  through  VII  of  this  report  are  concerned  with  the 
development  and  management  of  effective  ATD  training.  Chapter  VIII 
discussed  the  need  to  assess  ATD  training  programs  after  they  have  been 
developed  in  order  to  verify  that  they  are  effective.  This  final  chapter 
addresses  the  need  to  monitor  ongoing  ATD  training  and  training  support 
to  assure  that  the  level  of  training  effectiveness  established  through 
the  previous  efforts  is  maintained  over  time. 


FACTORS  IMPACTING  EFFECTIVENESS 

Many  kinds  of  factors  can  impact  ATD  training  by  reducing  its 
effectiveness.  Some  are  transitory,  such  as  the  temporary  disruptions 
that  occur  when  key  instructors  are  reassigned  or  when  the  device  is  out 
of  service  undergoing  modification.  Others  persist,  and,  if  not  attended 
to,  will  adversely  affect  training.  These  latter  problems  are  of  four 
principal  types:  (1)  changes  in  training  requirements  that  invalidate 
existing  training  programs;  (2)  modifications  to  the  aircraft  that  reduce 
its  correspondence  to  the  ATD;  (3)  inadequate  maintenance  support  which 
renders  portions  of  ATDs  inoperative;  and  (4)  deterioration  in  the  device 
training  program  itself.  Each  of  these  factors  is  discussed  below. 

Modified  Training  Requirements 

It  is  virtually  inevitable  that  changes  will  occur  from  time  to  time 
in  the  requirements  underlying  any  simulator  training  program.  These 
changes  may  occur  due  to  introduction  of  new  training  equipment, 
development  of  new  tactics,  or  safety  related  concerns.  When  a  new  part- 
task  trainer  is  acquired,  for  example,  some  training  previously 
conducted  in  a  flight  simulator  should  be  shifted  to  it,  permitting  tne 
simulator  to  be  used  for  additional  training.  When  mission  requirements 
or  tactics  change  because  there  is  a  need  to  nullify  improved  enemy  air 
defenses,  training  must  quickly  accommodate  this  need.  When  information 
from  accident  investigations  requires  changes  in  the  way  'aircrews  perform 
tasks,  established  training  practices  must  be  revised.  There  is  almost  a 

limitless  number  of  changes  that  can  take  place.  All  such  changes 

require  that  ATD  training  be  modified  in  some  manner  in  order  for  a 
previously  established  level  of  effectiveness  to  be  maintained. 

For  the  most  part,  it  was  found  that  the  need  to  adjust  or  modify 
ATD  training  following  changes  in  training  requirements  was  recognized  by 
the  personnel  responsible  for  ATD  training.  In  all  cases  at  the  units 
visited  during  the  present  survey,  action  had  been  completed  or  was 
reported  to  be  underway  to  adjust  ATD  training  as  necessary  to 

accommodate  all  changes  known  to  have  occurred  in  ATD  training 


In  some  aircraft,  changes  to  the  programming  of  on-board  computer  systems 
are  matters  of  particular  concern  and  frustration  to  ATD  users.  Unless 

modifications  are  made  in  the  ATOs  to  reflect  the  aircraft  hardware  and 

software  changes,  the  effectiveness  of  the  device  training  programs  may 
be  impaired. 

Nearly  all  of  the  ATDs  surveyed  during  the  present  project  needed 

updating,  although  the  need  appeared  greatest  with  devices  simulating  the 
newer  aircraft.  The  modifications  that  were  needed  to  update  the  ATDs  to 
the  current  configuration  of  the  aircraft  they  represented  included  the 
repositioning  of  existing  cockpit  displays  or  controls,  changing  the 
simulated  performance  characteristics  of  the  aircraft's  engines, 
simulating  a  different  radar  or  EW  control  panel,  and  reprogramming  the 
simulation  of  the  on-board  computer  system.  In  some  instances,  lack  of 
the  required  ATD  modifications  was  probably  of  minor  training  transfer 
consequence  and  could  be  compensated  to  a  large  extent  by  calling  the 

students'  attention  to  the  ATD-aircraft  discrepancy  as  described  in 
Chapter  IV.  In  other  instances,  the  consequences  were  significant.  In 
one,  modifications  had  been  made  to  an  aircraft's  weapons  system  computer 
that  resulted  in  changes  in  the  sequence  of  procedural  tasks  performed  in 
the  cockpit  under  high  workload  condition,  and  negative  transfer  to  the 
target  aircraft  almost  surely  resulted. 

The  failure  to  modify  ATDs  promptly  when  aircraft  are  modified  is  a 
major  problem.  Delays  i n  effecting  important  modifications  to  ATDs  were 
counted  in  years  at  some  units.  In  some  instances,  no  updating  of  the 
device  was  even  planned,  and  training  objectives  that  had  been  allocated 
to  ATDs  prior  to  aircraft  modification  had  been  reallocated  to  aircraft. 

There  are  a  number  of  reasons  for  the  existence  of  this  problem.  An 
obvious  one  is  the  need  for  funds  to  accomplish  the  device  modification 
In  mo s t  cases,  funds  for  modi  fications  made  to  aircraft  and  to  ATDs 
originate  from  different  budge*:  sources,  so  the  availability  of  funds  for 
the  former  does  not  assure  fjnds  for  the  latter.  One  exception  to  this 
situation  was  noted  during  the  survey,  however,  that  apparently  solved 
the  problem:  One  military  service  required  that  funding  for  all  projects 
that  involved  aircraft  modifications  cover  corresponding  modifications  to 
the  simulator  cor  that  aircraft. 

A  second  reason  involved  the  separation  of  responsibility  for 
modifying  the  two  vehicles.  The  usual  situation  is  that  one  agency  is 
responsible  for  modifying  aircraft,  while  another  for  modifying  ATDs--anc 
neither  of  these  has  responsibility  for  use  of  the  vehicle  for  training. 
As  a  sonsequence,  tnere  appears  to  be  inadequate  communication  between 
the  two,  ana  little  s-  no  effort  to  coordinate  tne  modi f ications  was 
noted.  In  fart.,  efforts  to  modify  the  ATD  usually  do  net  begin  until 
sometime  after  the  aircraft  modifications  have  been  completed  and  tested. 

Several  other  reasons  were  given  by  the  personnel  interviewed  for 
delays  in  obtaining  ATD  modifications.  These  reasons  includee  inaoeq.  ate 
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documentation  that  could  be  used  by  government  personnel  to  make  in-house 
device  modifications,  ATO  hardware  and  software  designs  that  were 
difficult  to  modify,  and  constraints  associated  with  the  contracting 
process  ("red  tape"). 

In  view  of  the  predictability  of  effects  of  aircraft  modifications, 
and  the  inherent  flexibility  of  the  performance  of  digital  computer- 
controlled  ATDs,  there  is  no  need  for  lengthy  delays  in  modifying 
simulators  to  reflect  aircraft  changes.  Clearly,  however,  such  delays 
are  inherent  in  present  approaches  to  the  management  of  requirements  for 
ATD  modifications.  It  is  believed  that  the  maintenance  of  ATD  training 
effectiveness  could  be  improved  if  more  attention  were  directed  to 
expediting  critical  modifications  to  ATDs. 

Inadequate  Device  Maintenance 

An  ATD  must  be  working  correctly  if  it  is  to  be  used  effectively. 
Therefore,  effective  maintenance  suppport  is  requisite  to  effective  ATD 
utilization  over  an  extended  period.  As  was  indicated  in  Chapter  VII, 
very  few  instances  of  lost  training  periods  noted  during  the  survey  were 
attributable  to  inadequate  maintenance  support.  Those  that  were 
identified  were  associated  with  old  and  possibly  worn  out  devices  or  new 
devices  that  were  still  undergoing  debugging  activities.  ho  examples 
were  reported  during  the  project  interviews  in  which  adequately  trained 
in-house  or  contract  maintenance  personnel  were  unavailable  within 
reasonable  time  periods  when  breakdown  occurred.  In  fact,  the  quality  of 
available  maintenance  support,  in  combination  with  the  reliability  of 
current  generation  flight  simulators,  typically  resulted  in  an  average 
loss  of  less  than  a  single  scheduled  training  period  per  werk. 

Unfortunately,  the  above  statements  concerning  the  availability  of 
ATDs  for  training,  while  correct,  tend  to  be  misleading.  There  were  many 
instances  when  ATDs  were  used  with  a  significant  subsystem  inoperative 
due  to  unavailabil ity  of  parts.  Further,  it  was  not  unusual  tc  find  that 
the  ATD  subsystem  in  question  had  been  "down  for  parts"  for  extended 
periods.  Motion  and  visual  subsystems  were  the  ones  most  commonly 
needing  parts,  but  problems  existed  with  other  simulator  subsystems  as 
well.  For  example,  cockpit  instruments  were  sometimes  inoperative  for 
want  of  replacement  spares,  and  one  device  reportedly  had  been  without  a 
part  for  its  Record/Playback  instructional  feature  literally  for  years. 

When  needed  component  spares  are  not  readily  available  or  when  any 
other  aspect  of  maintenance  reduces  the  availability  of  ari  ATD  subsystem 
for  training,  training  effectiveness  is  likely  to  be  reduced.  Mao> 
times,  the  device  continues  to  be  used--and  effectively,  too,  for 
training  that  is  not  dependent  upon  the  down  subsystem--but  its  overall 
contribution  to  aircrew  training  is  likely  degraded  beyond  that  reflected 
in  the  ATD  utilization  rate  data. 
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It  should  be  noted  that  efforts  are  being  made  by  ATD  procurement 
and  logistic  agencies  to  increase  the  quality  of  maintenance  support, 
even  beyond  the  high  levels  noted  during  the  present  project,  through  the 
conduct  of  product  improvement  programs  and  the  establishment  of 
centralized  ATD  maintenance  and  logistic  systems.  These  efforts  are 
concentrating  particularly  on  device  subsystems  with  relatively  high 
maintenance  personnel  support  requirements  and  problems  of  long  lead  time 
for  parts  replacement.  An  effort  by  the  Naval  Training  Equipment  Center 
to  analyze  maintenance  problems  associated  with  ATD  motion  systems  is 
noteworthy  in  that  regard. 

Several  mechanisms  already  exist  for  monitoring  the  maintenance 
status  of  ATDs.  Maintenance  logs  reflect  the  status  of  each  device  on  a 
continuing  basis  and  document  any  deficiencies  in  system  and  subsystem 
operation.  In  addition,  ATD  instructors  and  students  are  responsible  for 
reporting  device  aeficiencies  by  entering  the  information  in  a  "flight 
log"  maintained  for  that  purpose.  Both  the  maintenance  and  the  flight 
logs  can  serve  a  useful  purpose  in  determining  whether  training 
objectives  allocated  to  ATDs  can  continue  to  be  met  or  whether  ATD 
degradation  will  require  that  those  objectives  be  allocated  elsewhere. 

It  is  the  usual  practice  at  all  the  units  surveyed  for  the  assigned 
maintenance  personnel  to  operate  the  simulator  daily  for  the  purpose  of 
assessing  whether  its  responses  to  control  manipulations  are  consistent 
with  specified  performance  data.  All  of  the  more  sophisticated  ATDs 
employ  a  computer  to  assist  these  personnel  in  making  various  diagnostic 
checks  to  determine  whether  maintenance  is  needed.  Another  useful 
practice  frequently  followed  is  to  have  an  aircrewman  operate  the  device 
periodically  to  determine  whether  its  flight  responses  are  comparable  to 
those  of  the  aircraft  simulated.  Each  of  these  checks  can  serve  a  useful 
purpose  by  alerting  ATD  training  personnel  when  the  effectiveness  of 
their  training  programs  may  be  below  par. 

The  Air  Force  has  taken  steps  to  expand  its  procedures  for  verifying 
that  ATDs  continue  to  be  maintained  adequately.  A  "Simulator 
Certification"  program  is  currently  under  development  that  in  all 
likelihood  will  increase  the  effectiveness  of  these  monitoring  efforts. 
Information  from  that  program  should  be  monitored  by  training  personnel 
so  that,  as  ATD  deficiencies  are  detected,  adjustments  can  be  made  to 
assure  continued  overall  training  program  effectiveness,  e.g.,  by 
reallocating  to  the  aircraft  those  training  objectives  which  can  no 
longer  be  met  through  ATD  training  because  of  detected  deficiencies  in 
the  device. 

Training  Program  Deterioration 

The  fourth  type  of  factor  that  can  reduce  the  effectiveness  of  ATD 
training  involves  the  manner  in  which  the  device  is  used,  i.e.,  the  ATD 
training  program  itself.  Independently  of  whether  changes  attributable 
to  the  three  types  of  factors  discussed  above  have  occurred,  the  quality 
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of  the  training  itself  can  deteriorate.  When  this  occurs,  ir.tc-nc-.-o 
training  objectives  may  no  longer  be  met.  For  example,  if  Alt  training 
we  jins  to  concentrate  upon  actions  tnat  cannot  be  perf  ormed  in  the 
ai-craft,  key  instruction  and  guidance  is  no  longer  employed  during  pn  - 
training  session  briefinqs,  or  i nstructional  features  erase  to  be  used 
effectively,  the  necessary  ski'  ■  ,  niay  not  be  learned  within  the  time 
available  for  Alt  tra  imr.c. 

Changed  trair.i-r  ;  t  if  *'S  such  as  these,  when  they  occur,  car.  .« 
attributed  tc  mu  .>  r.  ,.s' s,  e.g.,  r-ijs ugnment  of  expe-i  .need 
instructors,  varijlm.  :  in  the  quality  of  student  input  cha’  u'd 
attitudes  toward  At !•  training.  and  lessened  management  attention  as  AT., 
newness  or  novelty  v.  -*ars  off.  A  training  program  previously  assess',  c  as 
effective  should  not  be  expected  tc  retain  its  effectiveness  as  these 
changes  occur. 

Gradual  ,  or  possibly  even  abrupt,  changes  in  ATD  training  programs 
such  as  those  cited  may  not  be  obvious,  particularly  to  personnel  in 
day-to-day  contact  with  the  training.  In  this  respect,  these  kinds  of 
changes  are  unlike  the  more  obvious  changes  in  training  requirements, 
aircraft  configuration,  and  device  availability  discussed  eari.'e-.  Even 
the  most  skilled  analysts  would  have  difficulty  monitoring  instructor 
experience,  student  quality,  student  and  instructor  attitudes,  etc.,  to 
determine  whether  a  change  had  occurred  that  would  likely  impact  ATD 
training  practices.  Nevertheless,  monitoring  of  such  factors  was  found 
to  be  the  principal  means  for  assessing  and  maintaining  consistency  in 
the  conduct  of  AfD  training  programs  among  the  units  surveyed. 

The  monitoring  efforts  noted  were  all  informal,  anc  most  were 
unsystematic.  They  consisted  of  the  following:  (1)  aperiodic 

discussions  among  ATD  instructors  of  problems  encountered  during  the 
conduct  of  training;  (2)  periodic  and  aperiodic  "standardisation'1 
meetings  in  which  a  section  chief  or  other  senior  instructor  discussed 
with  the  ATD  instructor's  syllabus  requirements,  instructional 
techniques,  and  scheduling  problems;  (3)  aperiodic  observation  of  ATD 
instruction  in  progress  by  the  same  individuals;  (4)  aperiodic  syl'labus 
review  by  ISO  team  representatives  or  ether  personnel  outside  the 
training  unit;  and  (S)  continuing  analysis  of  student  grade  slips  and 
program  records  o  y  a  variety  of  personnel  interested  in  training 
effectiveness.  These  efforts,  which  varied  in  format  and  frequency  from 
unit  tc  unit,  must  be  c-edited  for  the  levels  of  ATD  training 
effectiveness  currently  being  maintained. 

At  best,  however,  efforts  such  a  -  these  can  be  only  partially 
sati s factory .  While  they  will  detect  most  major  anomalies  in  the  way  in 
which  training  is  being  conducted,  they  are  largely  insensitive  to 
gradual  changes.  In  addition,  these  efforts  cannot  reliably  detect 
losses  ir  training  effectiveness  that  may  result  from  deliberate  changes 
in  course  content,  instructional  content,  or  schenu1 ing  procedures  that 
were  made  with  the  expectations  that  the  program  would  benefit  instead. 
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The  absence  of  a  reliable  mechanism  for  systematically  monitoring 
A T  Y  .  lining  effectiveness  on  a  continuing  basis  was  judged  to  be  an 
important  deficiency  in  the  management  of  the  A7D  training  surveyed 
dui in;  ine  present  project.  Regardless  of  the  best  of  intentions, 
lacking  surh  a  monitoring  mechanism,  some  deterioration  in  effectiveness 
should  DC-  expected  to  occur  over  time  in  any  ATb  training  program.  The 
consequence  is  likely  to  be  a  _ed  to  expand  other  elements  of  the 
overall  flight  training  system  to  compensate  for  the  loss  in  ATD  training 
effectiveness. 

It  should  be  rioted  that  ISf1  orocedures,  as  defined  in  AFP  50-58  and 
AFM  50-2,  explicitly  requite  Ine  monitoring  of  all  ISD-devel oped 
training.  Nore  of  the  AfD  training  programs  surveyed  during  the  current 
project  had  been  developed  as  a  nart  of  a  comprehensive  effort  to  "ISD" 
an  entire  flight  training  program,  although  ISD  teams  had  had  some  input 
to  most  of  these  programs.  In  "ime,  ISO  efforts  will  irclude  monitoring 
all  ongoing  AYD  training.  AFM  50  - ;  and  other  aocuments  defining  such 
requirements  do  not  specify  ho-v  the  monitoring  is  to  be  accomplished, 
however. 

The  following  section  of  this  report  describes  an  approach  to 
monitoring  ATD  training  that  woulc  provide  early  detection  of 
deterioration  in  training  or  loss  in  its  effectiveness.  With  the 
information  such  an  approach  could  provide,  deficiencies  could  be 
detected  and  corrected  early,  and  ATD  training  effectiveness  could  be 
more  easily  maintained. 


MONITORING  THE  ATD  TRAINING  PROCESS 

In  Chapter  VIII,  -:n  emphasis  was  placed  upon  the  measurement  of 
student  performance  as  a  means  of  assessing  ATD  training  program 
effectiveness.  That  is,  the  eff<  fineness  of  ATD  training  should  be 
determined  by  measuring  the  qualit >s  of  the  product  of  that  process. 
The  product  of  training,  of  course,  is  the  student  at  the  end  of  or 
subsequent  to  nis  training  in  the  device. 

Measuring  training  product  quality  is  analogous  to  determining  the 
effectiveness  of  a  manufacturing  process  by  measuring  the  qualities  of 
the  products  produced.  In  fact,  the  product  quality  control  systems 
established  by  industrial  manufacturers  provide  a  useful  model  for  the 
establishment  of  a  quality  control  system  to  monitor  the  effectiveness  of 
aircrew  training  programs  involving  ATDs . 

The  general  concept  of  a  quality  control  system  for  a  training 
program  has  been  discussed  by  Smith  <1965).  An  extensive  application  of 
the  concept  to  military  flight  training  has  been  described  by  Duffy  and 
Colgar  (1R63).  These  writers  outline  five  general  requirements  for  an 
effective  training  quality  control  system: 
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•  A  detailed  statement  of  training  objectives  based  on  job 
requi rements 

•  Accurate  and  appropriate  performance  measures 

•  Effective  communication  concerning  performance  of  students  on 
the  tests 

•  Effective  procedures  for  corrective  action,  if  necessary 

•  Supervisory  support 

Training  Objectives 


The  assessment  of  training  program  effecti veness  should  be  conducted 
in  terms  of  objective  standards  that  define  the  achievement  of  course 
objectives.  This  requirement  was  discussed  in  Chapter  VIII.  Since  the 
present  concern  is  the  maintenance  of  an  established  level  of 
performance,  the  applicable  standards  are  the  same  as  those  employed  in 
the  study  of  ATD  training  effectiveness  that  presumably  will  previously 
have  been  conducted. 

There  are,  however,  additional  concerns  regarding  an  ongoing 
monitoring  system  that  do  not  apply  when  the  effectiveness  of  the 
training  program  is  being  assessed  initially.  Because  the  initial 
assessment  study  is  a  one-time  activity,  it  may  be  feasible  to  employ 
comprehensive  performance  monitoring  and  data  analysis  techniques  or 
procedures  that  would  be  too  costly  to  consider  on  a  continuing  basis. 
Therefore,  it  may  not  be  practical  to  continue  to  assess  student 
performance  with  respect  to  all  of  the  training  objectives  assessed 
originally. 

Care  must  be  taken,  however,  to  assure  that  the  objectives  selected 
to  be  monitored  constitute  an  adequate  sample  of  the  performance  of 
interest.  There  could  be  a  problem  in  that  an  originally  adequate 
sample  may  not  remain  so  month  after  month.  There  would  be  a  tendency 
during  training  to  increase  emphasis  on  those  objectives  that  are 
evaluated  regularly,  and  possibly  to  neglect  those  that  are  never 
monitored.  This  problem  could  be  prevented  by  periodically  replacing 
some  of  the  objectives  sampled  on  one  occasion  with  others,  so  that  over 
time  most  if  not  all  significant  skills  will  have  been  sampled  and 
evaluated. 

The  adequacy  of  a  sample  at  a  given  time  must  be  left  to  the  expert 
judgment  of  the  personnel  involved.  However,  they  must  guard  against 
selecting  objectives  on  the  basis  of  convenience  or  ease  of  measurement. 
Confidence  in  the  adequacy  of  the  objectives  selected  with  respect  to 
overall  training  program  goals  is  a  better  basis  for  their  selection. 

Performance  Measurement 


Measurement  of  product  quality  is  the  source  of  the  feedback  about 
student  proficiency  that  is  the  cornerstone  of  a  training  quality  control 
system.  Because  this  information  defines  the  need  for  corrective  action, 
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it  is  critically  important  that  it  be  relevant  and  objective.  Rating 
scales  and  other  evaluative  indices,  e.g.,  daily  flight  grade  slips,  do 
not  provide  the  quality  of  information  needed  for  quality  control 
purposes.  How  to  obtain  measures  that  are  relevant  and  objective  is 
discused  in  Chapter  VIII. 

In  Chapter  VIII,  it  was  pointed  out  that  ATD  training  effectiveness 
ideally  should  be  determined  by  measuring  student  performance  in  the 
target  vehicle  after  the  ATD  training  has  been  completed.  The  same 
guidance  holds  with  respect  to  measurement  for  quality  control  purposes. 
However,  after  a  particular  ATD  training  program  has  been  validated 
through  a  transfer  of  training  study,  it  may  then  be  possible  to  measure 
student  performance  in  the  ATD  at  the  end  of  training  (or  at  times  during 
training  corresponding  to  the  times  at  which  performance  was  measured 
during  the  earlier  assessment  study),  and  to  use  those  data  for  quality 
control  purposes.  Such  uses  of  measures  of  ATD  performance  would  be 
justified  if  the  ATD  performances  selected  for  evaluation  had  been  shown 
in  the  earlier  evaluative  research  to  correlate  highly  with  corresponding 
performances  in  the  aircraft.  In  such  cases,  the  use  of  the  automatic 
performance  measurement  capabilities  designed  into  some  ATDs  would  be 
possible. 

Communication 


Data  obtained  by  measuring  student  proficiency  must  be  analyzed  and 
summarized  in  some  manner  so  that  the  strengths  and  weaknesses  of  the 
training  program  can  be  identified  and  communicated  to  management.  The 
information  to  be  extracted  from  these  data  relate  to  trends,  since 
deterioration  in  student  quality  can  be  a  slow  process  that  will  be 
apparent  only  over  a  period  of  time.  Unsatisfactory  performance  of 
individual  students  will  be  immediately  obvious,  of  course,  but  the 
performance  of  an  individual  reveals  little  about  the  quality  of  the 
training  program. 

Clear,  succi.ict,  and  meaningful  summaries  of  performance  data  must 
be  communicated  to  appropriate  instructional  and  managerial  personnel  so 
that  they  can  determine  whether  a  training  quality  maintenance  problem 
exists.  Judgments  of  the  adequacy  of  the  training  program  should  also  be 
communicated,  but  those  judgments  should  be  based  on  data,  not 
substituted  for  data.  The  data  summaries  and  accompanying  judgments  also 
should  provide  a  basis  for  decisions  concerning  corrective  action,  when 
such  is  required. 

Corrective  Action 


Quality  control  is  effective  only  when  corrective  action  can  be 
taken  quickly.  Such  action  is  best  achieved  through  the  judgment  and 
skill  of  experienced  training  program  development  personnel  and  highly 
competent  managers  of  ATD  training  programs.  The  action  taken  must 
reflect  a  thorough  understanding  of  the  performance  data  and  the 
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relationship  between  deficiencies  in  performance  reflected  in  those  data 
and  the  aspect  of  the  training  program  changed. 

As  with  changes  made  in  A  I'D  training  in  response  to  modified 
training  requirements,  the  corrective  actions  taken  should  be  determined 
through  a  systematic  analysis  which  takes  into  consideration  all  elements 
of  the  instructional  system,  and  the  rationale  for  the  existing  design  of 
these  elements.  Rather  than  short-term  "patches"  to  solve  a  specific 
problem,  the  changes  made  should  be  part  of  the  continuing  systematic 
development  of  the  instructional  system.  Additionally,  all  corrective 
actions  taken  in  training  should  be  formally  documented  to  promote  the 
consistent  implementations  of  the  changes,  and  to  make  toe  changes  and 
the  reasons  for  them  available  for  future  analysis. 

The  continueG  effectiveness  of  AID  training  is  dependent  upon  tne 

continued  application  of  corrective  action  when  required.  .o;n;,unicaticn 
of  information  about  product  quality  to  management  is  a  continuing 
proposition.  The  system  must  provide  for  regular  analyses  of  pei  forma  ice 
data  and  periodic  communication  of  tnose  data  to  management. 

Man agement _S  u  pport 

Strong  support  iron  all  levels  of  management  is  essential  to  mav 

the  quality  control  system  work.  Tc  secure  aria  maintain  tins  support, 
everyone  involved  must  have  a  clear  idea  of  tne  purpose  of  tne  system  ra¬ 
the  rationale  for  its  various  pares.  Key  management  ivrsonr  el  must 

understand  what  the  program  can  go  for  t;u  .•  in  terms  cf  ;u  1  ntai  ni  n3 
overall  mission  readiness,  and  they  must  be  con.  inced  of  tne  ,'o-p 
util  i  ty . 

In  addition,  responsibility  f lay -tu-day  inas  a- orient  oi  trie  s  t*  r.. 
must  be  focused.  Someone  must  be  *  <  enarge,  and  that  ir dividual  ->..t 
have  author  i  ty  to  schedule  performance  tests,  to  select  -  a  t i *. 
examiners,  and  to  call  performance  .'of  icie.ocies  tc  trie  at  cent'  cf  to.  . 
in  positions  to  take  necessary  corrective  action.  Conscript ; ; oi  icias 
and  procedures  for  corrective  actions  should  be  establ  isn  ;d  a  ieau  of 

time.  They  should  focus  or.  correct i  v,  deficiencies,  nut  singling  out 
individuals  to  blame.  In  fact,  if  punitive  action are  taxer,  against 
instructors  and  other  training  perso  nel ,  it  i ..  lively  tnui  the  quality 
control  process  itself  will  suffc-'. 

Reports  by  Duffy  and  Col  yon  1  1963),  $nj  yh  imtb),  and  Caro  ( 19e8) 
provide  guidance  for  managers  of  fl ight  training  ouality  control  systems 
that  applies  .e  a1!  these  issues  and  that  ;  u  i  -d  .*  ■ .  c  e  i readily  anaptabl*- 
to  ATD  training  situations. 


SUMMARY 

This  final  chapter  addressed  the  need  to  monitor  ongoing  AT  u 
training  and  training  support  f  assure  that  the  level  of  training 
effectiveness  established  throuar  previous  efforts  is  ’Maintained  over 
tine.  Information  «as  presented  oncoming  factors  that  can  act  to 
reduce  the  effectiveness  of  ingoing  t'b  training,  ana  mechani sms  for 
maintaining  training  effectiveness  were  discussed.  This  information  is 
summarized  below. 

•  ATP  training  should  be  revised  to  refl/ct  modi f i cat  ions  i  n  the 
rtgui • emonts  underlying  that  training.  Since  changes  occur  from  time  tc 
time  in  the  requi  mmc-nts  underlying  any  ATP  training  program,  eacn 
training  organization  should  have  a  mechanism  for  assuring  the  systematic 
revision  of  AID  train  I rg  to  accommodate  these  changes. 

•  AJJ.  adj profits  made  in  A~D  .V. lining  to accommodate  modified 

Lfajning.  requi  .•  yinen  ts  _  shou  1  d  be  a.  sc  an  ted .  The  absence  of  jp- to-date 
ATD  training  objectives  and  course  documents  tends  to  affect 
standardization  of  training  adversely. 

•  When  modi fi cat j on s  to  an  aircra ft  reduce  its  correspondence  to  an 
ATD,  the  device  should  be  modified  expedi  tiousi  y .  In  view  of  the 
predictabi 1 i ty  of  effects  of  aircraft  modification,  and  the  inherent 
flexibility  of  the  performance  of  digital  computer-control  led  ATDs,  tnere 
is  no  need  for  lengthy  delays  tc  occur  in  modifying  tne  simulator  to 
reject  aircraft  changes. 

•  Funding  lor all  projects that  involve aircraft  modifications 

s  hould  co- _e_r  _  correspon  ding  modi  f  ic  a  tiu>»  s  to the  ATUs  for  .  that  aircraft . 

Integration  or  funding  ensures  that  resources  are  available  for  the  Ale 
update  programs. 

•  ATDs should  be  maintained  so  that  all  ATP subsystems  function 

according  to  specif i cat ionSj  and  tnu s  not  restrict  training.  When 
inadequate  maintenance  reduces  the  availability  of  an  ATD  subsystem  for 
training,  the  overall  contribution  of  the  ATP  to  aircrew  training  is 
likely  to  be  degraded  beyond  that  reflected  in  utilization  rate  data. 

•  The  maintenance  statu.?,  of  the  AID  should  be  momtoreu  routinely. 
Tne  device  should  be  inspected  frequently  by  qualified  aircrew  personnel 
within  the  organization  as  well  as  by  maintenance  personnel. 

•  When _ deficiencies _ in __ATD  subsystems  a^e  detected,  adjustments 

s h o u i_d _ be  _ m a d  e  to  a ssure  _  continued _ overa  11 _ trai ning  program 

effectiveness .  The  device  can  continue  to  be  used  for  training  that  is 
not  dependent  on  a  down  subsystem,  but  ATa  training  objectives  that  can 
no  longer  be  met  through  ATD  training  because  of  defected  subsystem 
deficiencies  should  be  reallocated  to  the  aircraft. 
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•  The  quality  of  ATP  training  should  not  be  allowed  to  deteriorate 
over  time.  Some  factors  that  can  be  disruptive  of  ATD  training  (eTgT, 
turnover  of  personnel)  are  inherent  in  operational  training 
organizations,  and  their  occurrence  can  be  anticipated.  These  factors 
should  be  recognized,  and  steps  should  be  taken  to  prevent  any  disruptive 
effects  on  ATD  training. 

•  The  effectiveness  of  ATD  training  should  be  monitored  routinely. 

The  actual  effect  of  gradual,  or  possibly  even  abrupt,  changes  in  factors 
that  can  disrupt  ATD  training  may  not  always  be  obvious  to  personnel 

involved  in  that  training.  The  best  way  to  determine  whether  a  training 
program  is  continuing  to  be  effective  is  to  assess,  on  a  routine  basis, 
the  proficiency  of  the  graduates  of  that  program. 

•  Assessments  of  ATD  training  effectiveness  should  be  based  on 

detailed^  statements  of  training  objectives^  An  effective  ATD  training 
qual i ty  control  program  Ts  dependent  upon  the  existence  of  suitable 
objectives  for  that  training. 

•  The  objectives  selected  to  be  monitored  should  constitute  an 

adequate  sample  of  th e  performance  expected  of  the  graduate.  It  may  hot 
be  practical  or  cost  effective  to  assess  student  performance  routinely 
with  respect  to  all  of  the  training  objectives.  Nevertheless,  by 

selecting  samples  of  objectives  that  are  representative  of  all  training 
program  goals  and  by  varying  the  constitution  of  the  samples  over  time, 
adequacy  of  the  samples  used  for  monitoring  can  be  assured. 

•  Relevant  and  objective  performance  measures  should  be  employed  to 
obtain  data  for  qual i ty  control  purposes.  Rating  scales  and  other 
evaluative  indices  le.g.,  daily  flight  grade  slips)  do  not  provide  the 
quality  of  information  needed  for  quality  control  purposes. 

•  Clear,  succinct,  meaningful  summaries  of  performance  data  should 
be  comnunicated  to  appropriate  instructional  end  managerial  personnel. 
These  data  summaries  and  accompanying  judgments  as  to  the  adequacy  of 
training  provide  the  basis  for  the  identification  of  training  problems 
and  for  decisions  concerning  corrective  actions. 

•  Action  to  correct  training  problems  should  be  taken  quickly. 
Quality  control  is  most  effective  when  corrective  action  is  taken 
quickly. 

•  Corrective  action  to  remedy  deficiencies  in  ATD  training  should  be 
based  on  the  systematic  analysis  of  deficiencies  in  relation  to  alT 
elements  of  the  Instructional  system.  Adjustments  made  in  ATD  training 
to  correct  deficiencies  should  harmonize  with  the  overall  design  of  the 
aircrew  training  program. 
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•  Management  should  provide  adequate  support  to  the  Quality  control 
system.  To  secure  and  maintain  this  support,  everyone  involved  must  have 
a  clear  idea  of  the  purpose  of  the  system,  its  utility,  and  the  rationale 
for  its  various  parts.  Additionally,  someone  must  be  in  charge  and 
responsible  for  ensuring  that  all  of  the  quality  control  functions  are 
executed  properly. 
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A.  SIMULATOR  TRAINING  OBJECTIVES 


1.  Determine  origin  and  methods  of  development  of  simulator 
training  objectives  statements. 

2.  Identify  and  summarize  specific  training  objectives  which  are 
trained  only  in  the  simulator.  (Determine  why) 

3.  Identify  and  summarize  specific  training  objectives  which  are 
trained  only  in  the  aircraft.  (Determine  why) 

4.  Identify  and  summarize  specific  training  objectives  which  are 
trained  in  both  the  aircraft  and  simulator. 

5.  Locate  personnel  with  direct  knowledge  of  the  following: 

a.  Through  what  process  were  training  objectives  allocated  to 
simulator  vs.  other  media? 

b.  What  role  did  specific  simulator  design  features  play  in 
this  process? 

c.  How  was  engineering  fidelity  taken  into  consideration  in 
this  process? 

d.  What  other  considerations  were  involved  in  allocating 
objectives  to  simulator  training? 

6.  Describe  recent  changes  in  the  objectives  allocated  to  simulator 
training  and  indicate  reasons. 

7.  Identify  specific  simulator  design  changes  (e.g.,  updates  of 
outmoded  equipment  or  addition  of  new  capabilities)  that  would 
permit  additional  objectives  to  be  trained  and  describe  reasons. 

8.  To  what  extent  did  training  objectives  influence  initial  simula¬ 
tor  design  or  subsequent  device  modification? 

9.  Identify  specific  additional  training  objectives  that  could  be 
met  through  simulator  training  if  additional  training  time  were 
available. 

10.  What  were  the  qualifications  (e.g.,  education  and  experience 
other  than  in  aviation)  of  the  personnel  who  allocated 
objectives  to  simulators? 

11.  Surveyor's  assessment: 

a.  Validity  of  information  obtained 

b.  Rigor  of  application  of  systematic  method  to  objective 
allocation 

c.  Relevance  of  objectives  to  device  design 

d.  Relevance  of  objectives  to  training  programs 

e.  Other 

B.  SIMULATOR  TRAINING  METHODS  AND  TECHNIQUES 

1.  Are  there  any  personnel  involved  in  the  instructional  process 
who  are  not  flight  rated? 

2.  Are  peer  and/or  sel f- instructional  concepts  employed? 

3.  Is  train-to-proficiency  concept  employed? 

4.  Does  training  include  dynamic  observation? 

5.  Is  there  firm  continuity  in  instructional  staff 

a.  Within  simulator  training? 

b.  Across  simulator/other  devices/aircraft? 
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6.  How  are  instructional  events  sequenced  j 

a.  Within  simulator  training? 

b.  Across  simul ator/other  devices/aircraft?  j 

7.  How  is  training  in  the  simulator  procedural ly  or  methodolog-  j 

ically  different  from  training  in  the  aircraft?  ] 

a.  IndivTdu'al  j 

b.  Crew  I 

8.  Identify  and  characterize  major  elements  of  any  lesson  guides, 
plans  or  other  material  specifically  devoted  to  the  instruction 
of  students  in  the  simulator. 

9.  Describe  uses  of  simulators  for  proficiency  certification  (e.g., 
annual  instrument  checks)  and  other  proficiency  assessment 
programs  (e.g.,  operational  readiness  evaluations) 

10.  Characterize  the  impact  of  use  of  simulator  instructional 
features  on  simulator  utilization. 

a.  Identify  commonly  used  features. 

b.  Describe  perceived  impact  of  use  of  each. 

11.  Through  what  processes  are  individual  student  training  periods 
scheduled?  What  are  considerations? 

12.  Determine  the  extent  to  which  aircrew  training  (e.g.,  pilot, 
weapon  system  operator)  is  conducted  simultaneously  in  multiman 
or  linked  simulators. 

13.  Describe  innovative  uses  of  simul ator/simul ator  features. 

14.  Describe  perceived  problem/deficiencies  in  simulator  training 

methods.  j 

15.  How  is  simulator  training  quality  control  maintained? 

16.  Is  the  simulator  instructional  process  monitored?  How? 

17.  Performance  measurement  (impact  on  utilization). 

a.  How  is  performance  measured  (instrumentation  and  procedures)?  i 

b.  How  are  performance  criteria  established  (normative  or 
cri  terion-referen'-.ed)  ? 

c.  Satisfaction  of  instructors  and  students  with  performance 
measurement. 

d.  Impact  of  evaluatmn  of  performance  in  the  simulator. 

e.  Surveyor's  assessment  of  the  adequacy  of  evaluation  in  the 
simulator. 

18.  Describe  organizational/command  relationships  between  simulator 
training  and  other  training  activities. 

19.  Identify  simulator  training  management  factors  that  influence 
training  personnel  attitudes. 

20.  Characterize  the  nature  of  recent  significant  revisions  of  the 
training  program  which  modified  the  use  of  the  simulator. 

a.  Identify  reasons  for  the  revisions. 

b.  Identify  the  sources  of  requirements  for  these  revisions. 

c.  Identify  the  agency  or  individuals  who  actually  developed 
the  revised  syllabi  and  training  materials. 

21.  Surveyor's  assessment 

a.  Validity  of  information  reported 

b.  Utilization  innovations 

c.  Problems  noted 
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d.  Imp1 i cations  for  other  programs 

e.  Factors  constraining  effective  training 

f.  Effects  of  command./uni  t  utilization  philosophy 
Other 

C.  SIMULATION  TRAINING  EFFECTIVENESS 

1.  Foma!  tests  and  demonstration  of  effectiveness 

a.  Documentation  (obtain  copies) 

(1)  Published  reports 

(a)  Title 

(b)  Authors 

(c)  Date 

(d)  Publishing  organization 

(e)  Other  identifying  information 

(2)  Informal  reports 

(a)  Identifying  description 

(b)  Responsible  organization/individuals 

( c )  Date 

(dj  Other  identifying  information 

b.  Study  approach 

(1)  Experimental  design 

(2)  Subjects 

(3)  Independent  variables 
(.4)  Dependent  variables 

(5)  How  was  performance  measured 

c.  Results 

d.  Surveyor's  comments 

2.  Informal  tests  and  demonstrations  of  effectiveness 

a.  Procedures  employed 

b.  Independent  variables  considered 

c.  Dependent  variables  investigated 

d.  Findings 

e.  Responsible  organization/individuals 

f.  Surveyor's  assessment 

(1)  Rigor  of  study  efforts 

(2)  Validity  o*  findings 

(3)  Generality  of  results 

3.  Other  evidence  of  effectiveness 

4.  Effectiveness  assumptions  underlying  simulator  use 

a.  Assumptions 

b.  Bases  foi  assumptions 

5.  Benefits  of  simulator  training  (documented?) 

a.  Aircrew  performance 

b.  Safety 

c.  Economic 

d.  Other 
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6.  Effects  of  simulator  effectiveness  evidence 

a.  Subsequent  simulator  design/modification 

b.  Training  personnel  attitudes/expectations 

d.  Regulation/procedures 

e.  Other 

7.  Surveyor's  assessments 

a.  Simulator  effectiveness 

b.  Worth  of  simulator  ownership 

c.  Validity  of  user  perceptions 

d.  Problems  noted 

e.  Implications  for  each  program 

f.  Other 


D.  SIMULATOR  DEVELOPMENT  AND  INTRODUCTION 


1. 

2. 

3. 


4. 


5. 


6. 


7. 


8. 


9. 


Describe  source! s)  of  following  information. 

Initial  simulator  RFT  date. 

Describe  user  involvement  in: 

a.  ROC  preparation 

b.  Simulator  design 

c.  Simulator  development  (identify  developer) 

(1)  Design  review  conferences 

(2)  Mock-up 

(3)  In-plant  testing/checkout 

(4)  Other 

d.  On-site  testing 

e.  Initial  training  program  development 

f.  Subsequent  modifications 

Describe  factory  training  provided  and  summarize  perceived 
adequacy  of  this  training. 

a.  Instructors 

b.  Operators 

c.  Maintenance  personnel 

Characterize  user  attitudes  regarding  device's  training  values 
(related  to  changes  in  utilization  of  the  simulator.) 

a.  During  development 

b.  During  on-site  testing 

c.  At  RFT 


d.  One  year  after  RFT 

Describe  specific  efforts  designed  to  influence  user  attitudes/ 
expectations,  their  results. 

Characterize  user's  satisfaction  with  the  development  and 
introduction  processes. 

Characterize  user's  satisfaction  with  responsiveness  and 
adequacy  of  simulator  modification  procedures. 

Describe  user  recommendations  for  modified  processes  and 
identify  reasons  for  recommendations. 
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10.  Surveyor* s  assessment. 

a.  Validity  of  information  received 

b.  Problems  noted 

c.  Implications  for  future  developments 

>  d.  Other 

E.  SIMULATOR  UTILIZATION  RATES 

1.  Simulator  training  schedules 

a.  Total  no.  hrs.  of  simulator  training  per  individual 

b.  Total  no.  hrs.  of  inflight  training  per  individual 

c.  Hours/day,  days/week,  weeks/year 

d.  Make-up  schedules 

e.  Duration  of  individual  training  sessions;  identify  reasons 
for  selected  durations 

f.  How  coordinated  with  trainee's  other  activities 

2.  Factors  influencing  utilization  rates 

a.  Regulations  and  "guidance" 

b.  Training  syllabus  requirements 

c.  Schedule  considerations 

d.  Student  flow 

e.  Adequacy  of  maintenance  support 

f.  Attitudes  of  personnel 

g .  Other 

3.  Surveyor's  assessment 

a.  Validity  of  numbers  reported 

b.  Productivity  of  training  time  reported 

c.  Adequacy  of  maintenance  support 

d.  Role  of  regulations  and  "guidance" 

e.  Reasonableness  of  schedule  factors 

f.  Problems  noted 

g.  Implications  for  other  programs 

h .  Other 

F.  SIMULATOR  TRAINING  PERSONNEL 

1.  Instructor  (A/C  function:  _  _ ) 

a.  Typical  instructor  characteristics 

(1)  Rank 

(2)  Flight  experience 

(a)  Total  hours 

(b)  Hours  in  A/C  type 

(c)  Hours  In  combat 

(d)  Other  relevant  experience 

(3)  Instructing  experience 
(a)  Prior  flight 

(bj  Prior  flight  support 
(c)  Other 

b.  Selection  criteria/procedures 
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i  .  Hours  worked/' we  ok 

,  I)  Student  training  activities 

(2)  Unrelated  other  duties 
instructor/trainee  ratio 
Perceived  effects  of  assignment  or.  career 
{  Role  in  Simula tor/program  development 

Perceived  value  of  no j or  simulator  features 
i.l)  Met’ on 
(;•  l  Visual 
( i )  Sensor 
(4)  Navigatio" 

,5;  Crew  position  links 
; 6 )  Weapons  delivery 
(?)  Malfunctions 
IB)  Other 

h.  Perceived  adequacy  of  maintenance  support 

i.  Cr  ar-acte>'ize  attitudes  toward  simulators 

j.  .-.hat  factors  nopoar  to  influence  those  attitudes? 

Instructor  training 

a.  VC -related 

b .  Simulator-related 

(i)  Identify  and  characterize  major  elements  of  any 
courses,  training  materials,  or  OuT  supervision 
designed  to. 

ia',  help  the  instructor  in  the  operation  of  tne 
simul  ..tor, 

(b.  make  the  instructors  aware  of  tie  instructional 
features  and  training  potential  of  the  simulator; 
(o'  help  the  instructor  plan  his  instruction  in  tne 
simul ator ; 

(d)  ether 

\l)  Identify  and  characterize  any  instructor  job  aids. 

.  Otner 

e  "’erceived  worth/relevance  of  training  received 
Trainees  (A/C  function: 

a.  Describe  selection  erti eVi a/crocedures 

b.  Input  sources 

c.  Typical ‘prior  operational  experiences 

(1)  Years  in  service  and  rank 

(2)  Total  hours 

(3;  Hours  in  A/C-  type 

(4)  Hours  iri  combat 

(5)  Other  relevant  experience 

d  Perceived  value  of  simulator  training 
e.  Perceived  value  of  major  simulator  features 

(1)  Motion 

(2)  Visual 

(3)  Sensor 

(4)  Navigation 

(5)  Crew  position  links 


■6)  Weapons  delivery 

(7)  Malfunctions 

(8)  Other 

4.  Training  personnel  suggestions  for  change  ( idi  ot' fy 
suggestions' 

3.  Simulator  design 

b.  Training  program  content,  sequence,  com;-! otero-s 

c.  Training  schedules 

d.  Otner 

?  Surveyor's  assessment 

a.  Instructor  status'moral e 

b.  adequacy  of  instructor  training 

c.  Validity  of  instructor  train'.,;  assessments 
d  -  Probl  eais  no  .  >d 

e.  Implications  for  otnni  programs 

f.  Other 
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AID  -  aircrew  training  device. 

AUGMENTED  FEEDBACK  -  intrinsic  feedback  that  has  been  enhanced,  usually 
during  training,  to  make  it  more  evident. 

CCT  -  combat  crew  training. 

CPT  -  cockpit  procedures  trainer. 

CT  -  continuation  training:  training  conducted  routinely  in  operational 
squadrons  or  proficiency  training  conducted  periodical ly . 

COGNITIVE  PROCESSES  -  actions  that  involve  the  mental  processing  of 
information;  perceptions  and  language  are  typically  involved,  out 
cognition  may  be  either  consciously  or  unconsciously  directed. 

COST  EFFECTIVE NtlSS/EFF IC IE NCY  -  the  extent  to  which  objectives  (e.g., 
training  goals)  are  achieved  with  minimal  financial  cost. 

CUE  -  the  informational  content  of  one  or  more  stimuli  that  signifies 

the  status  of  a  system,  what  actions  should  be  taken,  or  the 
outcome  of  actions  or  events  (feedback). 

CUE  DEVELOPMENT  -  the  process  of  learning  to  interpret  stimuli,  i.e.,  to 
derive  pertinent  information  from  conditions  and  events  as  guides 
for  actions. 

DISCRIMINATION  -  the  recognition  that  one  cue  or  response  differs 
significantly  from  a  similar  cue  or  response  in  terms  of  its 

nature,  meaning,  or  effects.  Discriminations  are  the  building 
blocks  of  any  motor  or  cognitive  skill. 

FEEDBACK  -  information  regarding  tne  adequacy  or  inadequacy  cf  an  action 
(e.g.,  a  motor  response,  decision,  judgment,  etc.). 

FIDELITY  -  see  physical  fidelity,  task  fidelity,  realism,  psychol cgical 
real i sm . 

GENERALIZATION  -  the  use  of  previously  learned  cue  or  response  discrim¬ 
inations,  and  their  processing,  in  a  new  or  differing  context. 

GUIDANCE  -  directing  a  person  (usually  a  learner)  to  perceive,  think, 
or  act  in  a  particular  manner;  the  focus  is  usually  on  cues  to  be 

discriminated,  ways  of  interpreting  them,  or  specific  responses  to 

make . 

ICS  -  instructor/operator  station  for  an  ATD. 
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ISO  -  instructional  systems  development:  procedural  approaches  to  the 
analysis  of  training  requirements  and  development  of  training 
systems. 

INTERFERENCE  -  (1)  the  confusion  or  competition  of  one  cue  or  response 
with  another,  usually  similar,  cue  or  response;  (2)  the  disruption 
of  cue  and  response  discriminations  due  to  extraneous  events  that 
compete  for  the  performer's  attention. 

INTRINSIC  FEEDBACK  -  feedback  that  is  inherent  in,  or  arises  naturally 
from,  the  performance  of  an  action;  refers  especially  to  realistic 
effects  of  operational  task  performance  whether  in  an  aircraft  or  a 
simulator. 

KINESTHETIC  FEEDBACK  -  feedback  that  is  part  of  the  "feel"  of  ongoing 
movements,  such  as*' control  inputs,  that  is  used  to  guide  and 
monitor  the  action. 

KINESTHETIC  MEDIATION  -  the  interpretation  of  movement  cues  and  the 

N  monitoring  of  physical  actions  through  the  "feel"  of  the  actions. 

MASTERY  -  the  ability  to  perform  at  a  high  level  of  proficiency; 
generally,  elements  of  performance  will  have  become  more  or  less 
mechanical  with  various  superfluous  cognitive  aids  reduced  to  a 
minimum. 

MEDIATION  -  (1)  the  use  of  past  experience  to  provide  cue  meaning  to 
stimuli,  process  information  contained  in  cues,  and  select  and 
implement  a  response  or  action;  (2)  an  approach  to  explaining  or 
predicting  transfer  of  ATO-trained  skills  to  aircraft  performance 
that  is  based  on  (1). 

MOTIVATION  -  the  degree  of  intent  to  learn  or  perform  in  a  superior 
manner  as  evidenced  by  conscientious  involvement  in  learning  or 
performance. 

OFT  -  operational  flight  trainer. 

OVERLEARNING  -  superior  mastery  that  is  presumed  to  occur  when  practice 
of  a  skill  continues  substantially  beyond  the  point  when  profi¬ 
ciency  was  first  achieved. 

PHYSICAL  FIDELITY  -  the  degree  of  structural  or  dynamic  correspondence 
of  an  ATD  to  a  given  aircraft. 

PRACTICE  -  deliberate  participation  in  activities  for  the  purpose  of 
learning  or  mastering  skills  that  depend  on  the  thoughts  and  motor 
actions  involved  in  the  activities. 
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PSYCHOLOGICAL  REALISM  -  the  same  as  Realism,  except  that  this  term 
denotes  the  role  of  subjective  or  mediational  foundations  for 
realism,  regardless  of  the  degree  of  objective  equivalence  of  ATD 
and  aircraft  experiences. 

REALISM  -  the  extent  to  which  an  aircrewman's  experiences  in  an  ATD 
correspond  to  experiences  as  they  would  actually  occur  in  an 
aircraft  under  a  given  set  of  conditions. 

RESPONSE  -  any  motor,  perceptual,  or  mental  act  by  a  person;  as  used 
here,  response  generally  refers  to  an  element  of  an  overall  action 
as  opposed  to  the  overall  action  itself. 

RETENTION  -  the  capacity  to  remember  task  requirements,  and  perform 
accordingly,  after  a  lapse  of  time  during  which  the  task  was  not 
practiced. 

STRES  -  Simulator  Training  Requirements  and  Effectiveness  Study. 

SELF -GUIDANCE  -  guidance  that  a  person  has  learned  to  provide  himself, 
usually  through  understanding  of  the  interrelations  of  elements  of 
task  performance  and  its  relation  to  and  effects  on  the  system 
(e.g.,  aircraft)  involved. 

SIMLARITY  -  (1)  the  degree  of  physical  fidelity  or  correspondence  of 
stimuli  in  an  ATD,  or  task  actions  in  an  ATD,  to  their  counterparts 
in  a  target  aircraft;  (2)  an  approach  to  explaining  and  predicting 
transfer  of  ATD-trained  skills  to  aircraft  performance  that  is 
based  on  (1) . 

STIMULUS  -  an  external  or  internal  event  capable  of  affecting  a  sense 
receptor  and  of  acquiring  cue  value  (meaning). 

SUPPLEMENTAL  FEEDBACK  -  feedback  provided,  usually  during  training,  that 
is  not  of  itself  inherent  in  naturally  occurring  outcomes  of 
actions. 

TASK  FIDELITY  -  the  degree  of  correspondence  of  cues  and  responses 
accompanying  task  performance  in  an  ATD  to  those  characteristic  of 
analogous  performance  in  a  given  aircraft. 

TASK  INTEGRATION  -  the  process  of  combining  elements  of  task  performance 
into  an  effective  pattern  such  that  cue  interpretations  and  actions 
involved  are  sufficient  for  proficient  operational  performance. 

TRAINING  EFFECTIVENESS  -  the  thoroughness  with  which  the  objectives  of 
training  have  been  achieved,  regardless  of  training  efficiency. 

TRAINING  EFFICIENCY  -  the  extent  to  which  training  resources  (including 
time)  are  used  economically  while  achieving  training  effectiveness. 


TRANSFER  -  the  use  of  skills  learned  in  one  context  (e.g.,  an  ATD)  in  a 
substantially  different  context  (e.g.,  an  aircraft). 

TRANSITION  TRAINING  -  training  for  aircrewmen  transitioning  to  different 
operational  aircraft. 

UPT  -  undergraduate  pilot  training:  initial  pilot  qualification 

training. 

VERBAL  MEDIATION  -  mediation  that  depends  heavily  on  conscious  or 
unconscious  use  of  language  and  related  systems  of  symbolic 
processing. 

WST  -  weapon  systems  trainer. 
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